SherbyNet - /page/wikibook/entry.mu

<!DOCTYPE html>

<title>Feature selection</title>

</head>

Feature selection

</h1>

</header>

/* start https://en.wikipedia.org/ */

.mw-parser-output .ambox{border:1px solid #a2a9b1;border-left:10px solid #36c;background-color:#fbfbfb;box-sizing:border-box}.mw-parser-output .ambox+link+.ambox,.mw-parser-output .ambox+link+style+.ambox,.mw-parser-output .ambox+link+link+.ambox,.mw-parser-output .ambox+.mw-empty-elt+link+.ambox,.mw-parser-output .ambox+.mw-empty-elt+link+style+.ambox,.mw-parser-output .ambox+.mw-empty-elt+link+link+.ambox{margin-top:-1px}html body.mediawiki .mw-parser-output .ambox.mbox-small-left{margin:4px 1em 4px 0;overflow:hidden;width:238px;border-collapse:collapse;font-size:88%;line-height:1.25em}.mw-parser-output .ambox-speedy{border-left:10px solid #b32424;background-color:#fee7e6}.mw-parser-output .ambox-delete{border-left:10px solid #b32424}.mw-parser-output .ambox-content{border-left:10px solid #f28500}.mw-parser-output .ambox-style{border-left:10px solid #fc3}.mw-parser-output .ambox-move{border-left:10px solid #9932cc}.mw-parser-output .ambox-protection{border-left:10px solid #a2a9b1}.mw-parser-output .ambox .mbox-text{border:none;padding:0.25em 0.5em;width:100%}.mw-parser-output .ambox .mbox-image{border:none;padding:2px 0 2px 0.5em;text-align:center}.mw-parser-output .ambox .mbox-imageright{border:none;padding:2px 0.5em 2px 0;text-align:center}.mw-parser-output .ambox .mbox-empty-cell{border:none;padding:0;width:1px}.mw-parser-output .ambox .mbox-image-div{width:52px}@media(min-width:720px){.mw-parser-output .ambox{margin:0 10%}}@media print{body.ns-0 .mw-parser-output .ambox{display:none!important}}

/* end https://en.wikipedia.org/ */

</style>

/* start https://en.wikipedia.org/ */

.mw-parser-output .hatnote{font-style:italic}.mw-parser-output div.hatnote{padding-left:1.6em;margin-bottom:0.5em}.mw-parser-output .hatnote i{font-style:normal}.mw-parser-output .hatnote+link+.hatnote{margin-top:-0.5em}@media print{body.ns-0 .mw-parser-output .hatnote{display:none!important}}

/* end https://en.wikipedia.org/ */

</style><div role="note" class="hatnote navigation-not-searchable">Not to be confused with <a href="Feature_extraction" class="mw-redirect" title="Feature extraction">Feature extraction</a>.</div>

In machine learning, feature selection is the process of selecting a subset of relevant <a href="Feature_(machine_learning)" title="Feature (machine learning)">features</a> (variables, predictors) for use in model construction. Feature selection techniques are used for several reasons:

<ul><li>simplification of models to make them easier to interpret,<a href="#cite_note-islr-1">[1]</a></li>

<li>shorter training times,<a href="#cite_note-2">[2]</a></li>

<li>to avoid the <a href="Curse_of_dimensionality" title="Curse of dimensionality">curse of dimensionality</a>,<a href="#cite_note-3">[3]</a></li>

<li>improve the compatibility of the data with a certain learning model class,<a href="#cite_note-4">[4]</a></li>

<li>to encode inherent <a href="Symmetric_space" title="Symmetric space">symmetries</a> present in the input space.<a href="#cite_note-5">[5]</a><a href="#cite_note-6">[6]</a><a href="#cite_note-7">[7]</a><a href="#cite_note-8">[8]</a></li></ul>

The central premise when using feature selection is that data sometimes contains features that are redundant or irrelevant, and can thus be removed without incurring much loss of information.<a href="#cite_note-Bermingham-prolog-9">[9]</a> Redundancy and irrelevance are two distinct notions, since one relevant feature may be redundant in the presence of another relevant feature with which it is strongly correlated.<a href="#cite_note-guyon-intro-10">[10]</a>

<a href="Feature_extraction" class="mw-redirect" title="Feature extraction">Feature extraction</a> creates new features from functions of the original features, whereas feature selection finds a subset of the features. Feature selection techniques are often used in domains where there are many features and comparatively few samples (data points).

<div class="mw-heading mw-heading2"><h2 id="Introduction">Introduction</h2></div>

A feature selection algorithm can be seen as the combination of a search technique for proposing new feature subsets, along with an evaluation measure which scores the different feature subsets. The simplest algorithm is to test each possible subset of features finding the one which minimizes the error rate. This is an exhaustive search of the space, and is computationally intractable for all but the smallest of feature sets. The choice of evaluation metric heavily influences the algorithm, and it is these evaluation metrics which distinguish between the three main categories of feature selection algorithms: wrappers, filters and embedded methods.<a href="#cite_note-guyon-intro-10">[10]</a>

<ul><li>Wrapper methods use a predictive model to score feature subsets. Each new subset is used to train a model, which is tested on a hold-out set. Counting the number of mistakes made on that hold-out set (the error rate of the model) gives the score for that subset. As wrapper methods train a new model for each subset, they are very computationally intensive, but usually provide the best performing feature set for that particular type of model or typical problem.</li>

<li>Filter methods use a proxy measure instead of the error rate to score a feature subset. This measure is chosen to be fast to compute, while still capturing the usefulness of the feature set. Common measures include the <a href="Mutual_information" title="Mutual information">mutual information</a>,<a href="#cite_note-guyon-intro-10">[10]</a> the <a href="Pointwise_mutual_information" title="Pointwise mutual information">pointwise mutual information</a>,<a href="#cite_note-textcat-11">[11]</a> <a href="Pearson_product-moment_correlation_coefficient" class="mw-redirect" title="Pearson product-moment correlation coefficient">Pearson product-moment correlation coefficient</a>, <a href="Relief_(feature_selection)" title="Relief (feature selection)">Relief-based algorithms</a>,<a href="#cite_note-12">[12]</a> and inter/intra class distance or the scores of <a href="Statistical_hypothesis_testing" class="mw-redirect" title="Statistical hypothesis testing">significance tests</a> for each class/feature combinations.<a href="#cite_note-textcat-11">[11]</a><a href="#cite_note-13">[13]</a> Filters are usually less computationally intensive than wrappers, but they produce a feature set which is not tuned to a specific type of predictive model.<a href="#cite_note-14">[14]</a> This lack of tuning means a feature set from a filter is more general than the set from a wrapper, usually giving lower prediction performance than a wrapper. However the feature set doesn't contain the assumptions of a prediction model, and so is more useful for exposing the relationships between the features. Many filters provide a feature ranking rather than an explicit best feature subset, and the cut off point in the ranking is chosen via <a href="Cross-validation_(statistics)" title="Cross-validation (statistics)">cross-validation</a>. Filter methods have also been used as a preprocessing step for wrapper methods, allowing a wrapper to be used on larger problems. One other popular approach is the Recursive Feature Elimination algorithm,<a href="#cite_note-15">[15]</a> commonly used with <a href="Support_Vector_Machines" class="mw-redirect" title="Support Vector Machines">Support Vector Machines</a> to repeatedly construct a model and remove features with low weights.</li>

<li>Embedded methods are a catch-all group of techniques which perform feature selection as part of the model construction process. The exemplar of this approach is the <a href="Lasso_(statistics)" title="Lasso (statistics)">LASSO</a> method for constructing a linear model, which penalizes the regression coefficients with an L1 penalty, shrinking many of them to zero. Any features which have non-zero regression coefficients are 'selected' by the LASSO algorithm. Improvements to the LASSO include Bolasso which bootstraps samples;<a href="#cite_note-Bolasso-16">[16]</a> <a href="Elastic_net_regularization" title="Elastic net regularization">Elastic net regularization</a>, which combines the L1 penalty of LASSO with the L2 penalty of <a href="Ridge_regression" title="Ridge regression">ridge regression</a>; and FeaLect which scores all the features based on combinatorial analysis of regression coefficients.<a href="#cite_note-FeaLect-17">[17]</a> AEFS further extends LASSO to nonlinear scenario with autoencoders.<a href="#cite_note-18">[18]</a> These approaches tend to be between filters and wrappers in terms of computational complexity.</li></ul>

In traditional <a href="Regression_analysis" title="Regression analysis">regression analysis</a>, the most popular form of feature selection is <a href="Stepwise_regression" title="Stepwise regression">stepwise regression</a>, which is a wrapper technique. It is a <a href="Greedy_algorithm" title="Greedy algorithm">greedy algorithm</a> that adds the best feature (or deletes the worst feature) at each round. The main control issue is deciding when to stop the algorithm. In machine learning, this is typically done by <a href="Cross-validation_(statistics)" title="Cross-validation (statistics)">cross-validation</a>. In statistics, some criteria are optimized. This leads to the inherent problem of nesting. More robust methods have been explored, such as <a href="Branch_and_bound" title="Branch and bound">branch and bound</a> and piecewise linear network.

<div class="mw-heading mw-heading2"><h2 id="Subset_selection">Subset selection</h2></div>

Subset selection evaluates a subset of features as a group for suitability. Subset selection algorithms can be broken up into wrappers, filters, and embedded methods. Wrappers use a <a href="Search_algorithm" title="Search algorithm">search algorithm</a> to search through the space of possible features and evaluate each subset by running a model on the subset. Wrappers can be computationally expensive and have a risk of over fitting to the model. Filters are similar to wrappers in the search approach, but instead of evaluating against a model, a simpler filter is evaluated. Embedded techniques are embedded in, and specific to, a model.

Many popular search approaches use <a href="Greedy_algorithm" title="Greedy algorithm">greedy</a> <a href="Hill_climbing" title="Hill climbing">hill climbing</a>, which iteratively evaluates a candidate subset of features, then modifies the subset and evaluates if the new subset is an improvement over the old. Evaluation of the subsets requires a scoring <a href="Metric_(mathematics)" class="mw-redirect" title="Metric (mathematics)">metric</a> that grades a subset of features. Exhaustive search is generally impractical, so at some implementor (or operator) defined stopping point, the subset of features with the highest score discovered up to that point is selected as the satisfactory feature subset. The stopping criterion varies by algorithm; possible criteria include: a subset score exceeds a threshold, a program's maximum allowed run time has been surpassed, etc.

Alternative search-based techniques are based on <a href="Targeted_projection_pursuit" title="Targeted projection pursuit">targeted projection pursuit</a> which finds low-dimensional projections of the data that score highly: the features that have the largest projections in the lower-dimensional space are then selected.

Search approaches include:

<ul><li>Exhaustive<a href="#cite_note-19">[19]</a></li>

<li><a href="Best-first_search" title="Best-first search">Best first</a></li>

<li><a href="Simulated_annealing" title="Simulated annealing">Simulated annealing</a></li>

<li><a href="Genetic_algorithm" title="Genetic algorithm">Genetic algorithm</a><a href="#cite_note-20">[20]</a></li>

<li><a href="Greedy_algorithm" title="Greedy algorithm">Greedy</a> forward selection<a href="#cite_note-21">[21]</a><a href="#cite_note-22">[22]</a><a href="#cite_note-23">[23]</a></li>

<li>Greedy backward elimination</li>

<li><a href="Particle_swarm_optimization" title="Particle swarm optimization">Particle swarm optimization</a><a href="#cite_note-sciencedirect.com-24">[24]</a></li>

<li><a href="Targeted_projection_pursuit" title="Targeted projection pursuit">Targeted projection pursuit</a></li>

<li>Scatter search<a href="#cite_note-25">[25]</a><a href="#cite_note-26">[26]</a><a href="#cite_note-27">[27]</a></li>

<li><a href="Variable_neighborhood_search" title="Variable neighborhood search">Variable neighborhood search</a><a href="#cite_note-28">[28]</a><a href="#cite_note-29">[29]</a></li></ul>

Two popular filter metrics for classification problems are <a href="Correlation" title="Correlation">correlation</a> and <a href="Mutual_information" title="Mutual information">mutual information</a>, although neither are true <a href="Metric_(mathematics)" class="mw-redirect" title="Metric (mathematics)">metrics</a> or 'distance measures' in the mathematical sense, since they fail to obey the <a href="Triangle_inequality" title="Triangle inequality">triangle inequality</a> and thus do not compute any actual 'distance' – they should rather be regarded as 'scores'. These scores are computed between a candidate feature (or set of features) and the desired output category. There are, however, true metrics that are a simple function of the mutual information;<a href="#cite_note-30">[30]</a> see <a href="Mutual_information#Metric" title="Mutual information">here</a>.

Other available filter metrics include:

<ul><li>Class separability

<ul><li>Error probability</li>

<li>Inter-class distance</li>

<li>Probabilistic distance</li>

<li><a href="Entropy_(Information_theory)" class="mw-redirect" title="Entropy (Information theory)">Entropy</a></li></ul></li>

<li>Consistency-based feature selection</li>

<li>Correlation-based feature selection</li></ul>

<div class="mw-heading mw-heading2"><h2 id="Optimality_criteria">Optimality criteria</h2></div>

The choice of optimality criteria is difficult as there are multiple objectives in a feature selection task. Many common criteria incorporate a measure of accuracy, penalised by the number of features selected. Examples include <a href="Akaike_information_criterion" title="Akaike information criterion">Akaike information criterion</a> (AIC) and <a href="Mallows's_Cp" title="Mallows's Cp">Mallows's Cp</a>, which have a penalty of 2 for each added feature. AIC is based on <a href="Information_theory" title="Information theory">information theory</a>, and is effectively derived via the <a href="Maximum_entropy_principle" class="mw-redirect" title="Maximum entropy principle">maximum entropy principle</a>.<a href="#cite_note-31">[31]</a><a href="#cite_note-32">[32]</a>

Other criteria are <a href="Bayesian_information_criterion" title="Bayesian information criterion">Bayesian information criterion</a> (BIC), which uses a penalty of <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\sqrt {\log {n}}}}">

<msqrt>

</mrow>

</msqrt>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\sqrt {\log {n}}}}</annotation>

</semantics>

</math><img src="./a58498f9cef85b6f5131e178f2b442273622c2c4.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.171ex; width:7.077ex; height:3.509ex;" alt="{\displaystyle {\sqrt {\log {n}}}}" loading="lazy"> for each added feature, <a href="Minimum_description_length" title="Minimum description length">minimum description length</a> (MDL) which asymptotically uses <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\sqrt {\log {n}}}}">

<msqrt>

</mrow>

</msqrt>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\sqrt {\log {n}}}}</annotation>

</semantics>

</math><img src="./a58498f9cef85b6f5131e178f2b442273622c2c4.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.171ex; width:7.077ex; height:3.509ex;" alt="{\displaystyle {\sqrt {\log {n}}}}" loading="lazy">, <a href="Bonferroni_correction" title="Bonferroni correction">Bonferroni</a> / RIC which use <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\sqrt {2\log {p}}}}">

<msqrt>

</mrow>

</msqrt>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\sqrt {2\log {p}}}}</annotation>

</semantics>

</math><img src="./800683652ed7965b451c232140f94a752ecb32aa.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.171ex; width:8.402ex; height:3.509ex;" alt="{\displaystyle {\sqrt {2\log {p}}}}" loading="lazy">, maximum dependency feature selection, and a variety of new criteria that are motivated by <a href="False_discovery_rate" title="False discovery rate">false discovery rate</a> (FDR), which use something close to <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\sqrt {2\log {\frac {p}{q}}}}}">

<msqrt>

<mfrac>

</mfrac>

</mrow>

</msqrt>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\sqrt {2\log {\frac {p}{q}}}}}</annotation>

</semantics>

</math><img src="./89c49d976f1a889749bfc320ff257ef1893f8efa.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -2.838ex; width:9.238ex; height:6.343ex;" alt="{\displaystyle {\sqrt {2\log {\frac {p}{q}}}}}" loading="lazy">. A maximum <a href="Entropy_rate" title="Entropy rate">entropy rate</a> criterion may also be used to select the most relevant subset of features.<a href="#cite_note-33">[33]</a>

<div class="mw-heading mw-heading2"><h2 id="Structure_learning">Structure learning</h2></div>

Filter feature selection is a specific case of a more general paradigm called <a href="Structured_prediction" title="Structured prediction">structure learning</a>. Feature selection finds the relevant feature set for a specific target variable whereas structure learning finds the relationships between all the variables, usually by expressing these relationships as a graph. The most common structure learning algorithms assume the data is generated by a <a href="Bayesian_Network" class="mw-redirect" title="Bayesian Network">Bayesian Network</a>, and so the structure is a <a href="Directed_graph" title="Directed graph">directed</a> <a href="Graphical_model" title="Graphical model">graphical model</a>. The optimal solution to the filter feature selection problem is the <a href="Markov_blanket" title="Markov blanket">Markov blanket</a> of the target node, and in a Bayesian Network, there is a unique Markov Blanket for each node.<a href="#cite_note-34">[34]</a>

<div class="mw-heading mw-heading2"><h2 id="Information_Theory_Based_Feature_Selection_Mechanisms">Information Theory Based Feature Selection Mechanisms</h2></div>

There are different Feature Selection mechanisms around that utilize <a href="Mutual_information" title="Mutual information">mutual information</a> for scoring the different features. They usually use all the same algorithm:

<ol><li>Calculate the <a href="Mutual_information" title="Mutual information">mutual information</a> as score for between all features (<math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle f_{i}\in F}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f_{i}\in F}</annotation>

</semantics>

</math><img src="./d6bd4141447552abd693e45c641b8f379461addf.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:6.52ex; height:2.509ex;" alt="{\displaystyle f_{i}\in F}" loading="lazy">) and the target class (c)</li>

<li>Select the feature with the largest score (e.g. <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\underset {f_{i}\in F}{\operatorname {argmax} }}(I(f_{i},c))}">

<mi>argmax</mi>

<mrow>

<msub>

</mrow>

</msub>

</mrow>

</munder>

</mrow>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\underset {f_{i}\in F}{\operatorname {argmax} }}(I(f_{i},c))}</annotation>

</semantics>

</math><img src="./e5c03a6b943d61123917490ecd5dda3da6777723.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -3.005ex; width:16.332ex; height:5.009ex;" alt="{\displaystyle {\underset {f_{i}\in F}{\operatorname {argmax} }}(I(f_{i},c))}" loading="lazy">) and add it to the set of selected features (S)</li>

<li>Calculate the score which might be derived from the <a href="Mutual_information" title="Mutual information">mutual information</a></li>

<li>Select the feature with the largest score and add it to the set of select features (e.g. <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\underset {f_{i}\in F}{\operatorname {argmax} }}(I_{derived}(f_{i},c))}">

<mi>argmax</mi>

<mrow>

<msub>

</mrow>

</msub>

</mrow>

</munder>

</mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\underset {f_{i}\in F}{\operatorname {argmax} }}(I_{derived}(f_{i},c))}</annotation>

</semantics>

</math><img src="./4bbbb805e14af56b8ed0bfa3c3471581c6dd5256.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -3.005ex; width:21.774ex; height:5.009ex;" alt="{\displaystyle {\underset {f_{i}\in F}{\operatorname {argmax} }}(I_{derived}(f_{i},c))}" loading="lazy">)</li>

<li>Repeat 3. and 4. until a certain number of features is selected (e.g. <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle |S|=l}">

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle |S|=l}</annotation>

</semantics>

</math><img src="./21e03445b7f93f6638c37329b2ee06a323e3c550.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:6.585ex; height:2.843ex;" alt="{\displaystyle |S|=l}" loading="lazy">)</li></ol>

The simplest approach uses the <a href="Mutual_information" title="Mutual information">mutual information</a> as the "derived" score.<a href="#cite_note-Brown-35">[35]</a>

However, there are different approaches, that try to reduce the redundancy between features.

<div class="mw-heading mw-heading3"><h3 id="Minimum-redundancy-maximum-relevance_(mRMR)_feature_selection">Minimum-redundancy-maximum-relevance (mRMR) feature selection</h3></div>

Peng et al.<a href="#cite_note-36">[36]</a> proposed a feature selection method that can use either mutual information, correlation, or distance/similarity scores to select features. The aim is to penalise a feature's relevancy by its redundancy in the presence of the other selected features. The relevance of a feature set S for the class c is defined by the average value of all mutual information values between the individual feature fi and the class c as follows:

<mfrac>

<mrow>

</mrow>

</mrow>

</mfrac>

</mrow>

<msub>

</mrow>

</msub>

</mrow>

</munder>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle D(S,c)={\frac {1}{|S|}}\sum _{f_{i}\in S}I(f_{i};c)}</annotation>

</semantics>

</math><img src="./d9c1631b388c22ad3ff32d1b7ef1a3f03620dfb4.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -3.505ex; width:25.323ex; height:6.843ex;" alt="{\displaystyle D(S,c)={\frac {1}{|S|}}\sum _{f_{i}\in S}I(f_{i};c)}" loading="lazy">.</dd></dl>

The redundancy of all features in the set S is the average value of all mutual information values between the feature fi and the feature fj:

<mfrac>

<mrow>

</mrow>

<msup>

</mrow>

</mrow>

</msup>

</mrow>

</mfrac>

</mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</munder>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle R(S)={\frac {1}{|S|^{2}}}\sum _{f_{i},f_{j}\in S}I(f_{i};f_{j})}</annotation>

</semantics>

</math><img src="./5f1951ebdc5225c2d7871e467f38ab4532a57355.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -3.838ex; width:27.195ex; height:7.176ex;" alt="{\displaystyle R(S)={\frac {1}{|S|^{2}}}\sum _{f_{i},f_{j}\in S}I(f_{i};f_{j})}" loading="lazy"></dd></dl>

The mRMR criterion is a combination of two measures given above and is defined as follows:

</mrow>

</mrow>

</munder>

<mrow>

<mrow>

<mfrac>

<mrow>

</mrow>

</mrow>

</mfrac>

</mrow>

<msub>

</mrow>

</msub>

</mrow>

</munder>

<msub>

</mrow>

</msub>

<mfrac>

<mrow>

</mrow>

<msup>

</mrow>

</mrow>

</msup>

</mrow>

</mfrac>

</mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</munder>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \mathrm {mRMR} =\max _{S}\left[{\frac {1}{|S|}}\sum _{f_{i}\in S}I(f_{i};c)-{\frac {1}{|S|^{2}}}\sum _{f_{i},f_{j}\in S}I(f_{i};f_{j})\right].}</annotation>

</semantics>

</math><img src="./3eec7b98cd9e6fc9b3b61c0ac4712a16379c8859.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -3.838ex; width:56.25ex; height:8.676ex;" alt="{\displaystyle \mathrm {mRMR} =\max _{S}\left[{\frac {1}{|S|}}\sum _{f_{i}\in S}I(f_{i};c)-{\frac {1}{|S|^{2}}}\sum _{f_{i},f_{j}\in S}I(f_{i};f_{j})\right].}" loading="lazy"></dd></dl>

Suppose that there are n full-set features. Let xi be the set membership <a href="Indicator_function" title="Indicator function">indicator function</a> for feature fi, so that xi=1 indicates presence and xi=0 indicates absence of the feature fi in the globally optimal feature set. Let <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle c_{i}=I(f_{i};c)}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle c_{i}=I(f_{i};c)}</annotation>

</semantics>

</math><img src="./05781826fbac546d7d5d9b93648d3fd09f23e66f.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:11.866ex; height:2.843ex;" alt="{\displaystyle c_{i}=I(f_{i};c)}" loading="lazy"> and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle a_{ij}=I(f_{i};f_{j})}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle a_{ij}=I(f_{i};f_{j})}</annotation>

</semantics>

</math><img src="./45b2ee932ee77d0349d578b5c60d1e9adeb6d1b9.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:13.808ex; height:3.009ex;" alt="{\displaystyle a_{ij}=I(f_{i};f_{j})}" loading="lazy">. The above may then be written as an optimization problem:

</mrow>

<msup>

</mrow>

</msup>

</mrow>

</munder>

<mrow>

<mrow>

<mfrac>

<mrow>

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

<mrow>

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

</mrow>

</mfrac>

</mrow>

<mfrac>

<mrow>

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

<mrow>

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

<msup>

</mrow>

</msup>

</mrow>

</mfrac>

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \mathrm {mRMR} =\max _{x\in \{0,1\}^{n}}\left[{\frac {\sum _{i=1}^{n}c_{i}x_{i}}{\sum _{i=1}^{n}x_{i}}}-{\frac {\sum _{i,j=1}^{n}a_{ij}x_{i}x_{j}}{(\sum _{i=1}^{n}x_{i})^{2}}}\right].}</annotation>

</semantics>

</math><img src="./0baef01e8c550ba917099a82e0ac43e826f59d37.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -3.171ex; width:49.606ex; height:7.509ex;" alt="{\displaystyle \mathrm {mRMR} =\max _{x\in \{0,1\}^{n}}\left[{\frac {\sum _{i=1}^{n}c_{i}x_{i}}{\sum _{i=1}^{n}x_{i}}}-{\frac {\sum _{i,j=1}^{n}a_{ij}x_{i}x_{j}}{(\sum _{i=1}^{n}x_{i})^{2}}}\right].}" loading="lazy"></dd></dl>

The mRMR algorithm is an approximation of the theoretically optimal maximum-dependency feature selection algorithm that maximizes the mutual information between the joint distribution of the selected features and the classification variable. As mRMR approximates the combinatorial estimation problem with a series of much smaller problems, each of which only involves two variables, it thus uses pairwise joint probabilities which are more robust. In certain situations the algorithm may underestimate the usefulness of features as it has no way to measure interactions between features which can increase relevancy. This can lead to poor performance<a href="#cite_note-Brown-35">[35]</a> when the features are individually useless, but are useful when combined (a pathological case is found when the class is a <a href="Parity_function" title="Parity function">parity function</a> of the features). Overall the algorithm is more efficient (in terms of the amount of data required) than the theoretically optimal max-dependency selection, yet produces a feature set with little pairwise redundancy.

mRMR is an instance of a large class of filter methods which trade off between relevancy and redundancy in different ways.<a href="#cite_note-Brown-35">[35]</a><a href="#cite_note-docs.google-37">[37]</a>

<div class="mw-heading mw-heading3"><h3 id="Quadratic_programming_feature_selection">Quadratic programming feature selection</h3></div>

mRMR is a typical example of an incremental greedy strategy for feature selection: once a feature has been selected, it cannot be deselected at a later stage. While mRMR could be optimized using floating search to reduce some features, it might also be reformulated as a global <a href="Quadratic_programming" title="Quadratic programming">quadratic programming</a> optimization problem as follows:<a href="#cite_note-QPFS-38">[38]</a>

</mrow>

</mrow>

</munder>

<mrow>

<mrow>

<msup>

</mrow>

</mrow>

</msup>

</mrow>

<msup>

</mrow>

</mrow>

</msup>

</mrow>

</mrow>

</mstyle>

</mrow>

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \mathrm {QPFS} :\min _{\mathbf {x} }\left\{\alpha \mathbf {x} ^{T}H\mathbf {x} -\mathbf {x} ^{T}F\right\}\quad {\mbox{s.t.}}\ \sum _{i=1}^{n}x_{i}=1,x_{i}\geq 0}</annotation>

</semantics>

</math><img src="./b051af44ce64bfc8883ed1a859b050f40e48433c.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -3.005ex; width:54.604ex; height:6.843ex;" alt="{\displaystyle \mathrm {QPFS} :\min _{\mathbf {x} }\left\{\alpha \mathbf {x} ^{T}H\mathbf {x} -\mathbf {x} ^{T}F\right\}\quad {\mbox{s.t.}}\ \sum _{i=1}^{n}x_{i}=1,x_{i}\geq 0}" loading="lazy"></dd></dl>

where <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle F_{n\times 1}=[I(f_{1};c),\ldots ,I(f_{n};c)]^{T}}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msup>

</mrow>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle F_{n\times 1}=[I(f_{1};c),\ldots ,I(f_{n};c)]^{T}}</annotation>

</semantics>

</math><img src="./e655a9d669fdf3ca7c6572563f8b5d1c1d7af44e.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:30.368ex; height:3.176ex;" alt="{\displaystyle F_{n\times 1}=[I(f_{1};c),\ldots ,I(f_{n};c)]^{T}}" loading="lazy"> is the vector of feature relevancy assuming there are n features in total, <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle H_{n\times n}=[I(f_{i};f_{j})]_{i,j=1\ldots n}}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle H_{n\times n}=[I(f_{i};f_{j})]_{i,j=1\ldots n}}</annotation>

</semantics>

</math><img src="./e5d1966a8fa8bd4894b5d768dcfc4da9b1caa9de.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:24.756ex; height:3.009ex;" alt="{\displaystyle H_{n\times n}=[I(f_{i};f_{j})]_{i,j=1\ldots n}}" loading="lazy"> is the matrix of feature pairwise redundancy, and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mathbf {x} _{n\times 1}}">

<msub>

</mrow>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \mathbf {x} _{n\times 1}}</annotation>

</semantics>

</math><img src="./dda623ce1d275ad19f1e706cbf92eb170b5d58e2.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:4.73ex; height:2.009ex;" alt="{\displaystyle \mathbf {x} _{n\times 1}}" loading="lazy"> represents relative feature weights. QPFS is solved via quadratic programming. It is recently shown that QFPS is biased towards features with smaller entropy,<a href="#cite_note-CMI-39">[39]</a> due to its placement of the feature self redundancy term <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle I(f_{i};f_{i})}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle I(f_{i};f_{i})}</annotation>

</semantics>

</math><img src="./5df783cb3f54fdc1ee7b140fe1baa65a98416dc0.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:7.893ex; height:2.843ex;" alt="{\displaystyle I(f_{i};f_{i})}" loading="lazy"> on the diagonal of H.

<div class="mw-heading mw-heading3"><h3 id="Conditional_mutual_information">Conditional mutual information</h3></div>

Another score derived for the mutual information is based on the conditional relevancy:<a href="#cite_note-CMI-39">[39]</a>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</munder>

<mrow>

<mrow>

<msup>

</mrow>

</mrow>

</msup>

</mrow>

</mrow>

</mstyle>

</mrow>

</mrow>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \mathrm {SPEC_{CMI}} :\max _{\mathbf {x} }\left\{\mathbf {x} ^{T}Q\mathbf {x} \right\}\quad {\mbox{s.t.}}\ \|\mathbf {x} \|=1,x_{i}\geq 0}</annotation>

</semantics>

</math><img src="./6d51c8d485a74228babd3d02ebd73df91c816ec8.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.838ex; width:46.892ex; height:4.176ex;" alt="{\displaystyle \mathrm {SPEC_{CMI}} :\max _{\mathbf {x} }\left\{\mathbf {x} ^{T}Q\mathbf {x} \right\}\quad {\mbox{s.t.}}\ \|\mathbf {x} \|=1,x_{i}\geq 0}" loading="lazy"></dd></dl>

where <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle Q_{ii}=I(f_{i};c)}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle Q_{ii}=I(f_{i};c)}</annotation>

</semantics>

</math><img src="./85553caf16dbbb8b5dde648bb9deace82c856381.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:13.265ex; height:2.843ex;" alt="{\displaystyle Q_{ii}=I(f_{i};c)}" loading="lazy"> and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle Q_{ij}=(I(f_{i};c|f_{j})+I(f_{j};c|f_{i}))/2,i\neq j}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

<msub>

</mrow>

</msub>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle Q_{ij}=(I(f_{i};c|f_{j})+I(f_{j};c|f_{i}))/2,i\neq j}</annotation>

</semantics>

</math><img src="./cbd8dfb5749085c2245f468617578f7bfdd85b04.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:38.595ex; height:3.009ex;" alt="{\displaystyle Q_{ij}=(I(f_{i};c|f_{j})+I(f_{j};c|f_{i}))/2,i\neq j}" loading="lazy">.

An advantage of SPECCMI is that it can be solved simply via finding the dominant eigenvector of Q, thus is very scalable. SPECCMI also handles second-order feature interaction.

<div class="mw-heading mw-heading3"><h3 id="Joint_mutual_information">Joint mutual information</h3></div>

In a study of different scores Brown et al.<a href="#cite_note-Brown-35">[35]</a> recommended the joint mutual information<a href="#cite_note-40">[40]</a> as a good score for feature selection. The score tries to find the feature, that adds the most new information to the already selected features, in order to avoid redundancy. The score is formulated as follows:

<mtr>

<mtd>

<msub>

</mrow>

</msub>

</mtd>

<mtd>

<msub>

</mrow>

</msub>

</mrow>

</munder>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

<msub>

</mrow>

</msub>

</mtd>

</mtr>

<mtr>

<mtd>

<msub>

</mrow>

</msub>

</mrow>

</munder>

</mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</mrow>

</mtd>

</mtr>

</mtable>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\begin{aligned}JMI(f_{i})&=\sum _{f_{j}\in S}(I(f_{i};c)+I(f_{i};c|f_{j}))\\&=\sum _{f_{j}\in S}{\bigl [}I(f_{j};c)+I(f_{i};c)-{\bigl (}I(f_{i};f_{j})-I(f_{i};f_{j}|c){\bigr )}{\bigr ]}\end{aligned}}}</annotation>

</semantics>

</math><img src="./3c44f7ace0374e11b551d8a9f254513a1cb431d1.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -5.671ex; width:61.028ex; height:12.509ex;" alt="{\displaystyle {\begin{aligned}JMI(f_{i})&=\sum _{f_{j}\in S}(I(f_{i};c)+I(f_{i};c|f_{j}))\\&=\sum _{f_{j}\in S}{\bigl [}I(f_{j};c)+I(f_{i};c)-{\bigl (}I(f_{i};f_{j})-I(f_{i};f_{j}|c){\bigr )}{\bigr ]}\end{aligned}}}" loading="lazy"></dd></dl>

The score uses the <a href="Conditional_mutual_information" title="Conditional mutual information">conditional mutual information</a> and the <a href="Mutual_information" title="Mutual information">mutual information</a> to estimate the redundancy between the already selected features (<math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle f_{j}\in S}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f_{j}\in S}</annotation>

</semantics>

</math><img src="./2921b0582199be2a706ad7e78c587cdfc52dd70e.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:6.389ex; height:2.843ex;" alt="{\displaystyle f_{j}\in S}" loading="lazy">) and the feature under investigation (<math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle f_{i}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle f_{i}}</annotation>

</semantics>

</math><img src="./65da883ca3d16b461e46c94777b0d9c4aa010e79.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:1.939ex; height:2.509ex;" alt="{\displaystyle f_{i}}" loading="lazy">).

<div class="mw-heading mw-heading2"><h2 id="Hilbert-Schmidt_Independence_Criterion_Lasso_based_feature_selection">Hilbert-Schmidt Independence Criterion Lasso based feature selection</h2></div>

For high-dimensional and small sample data (e.g., dimensionality > 105 and the number of samples < 103), the Hilbert-Schmidt Independence Criterion Lasso (HSIC Lasso) is useful.<a href="#cite_note-HSICLasso-41">[41]</a> HSIC Lasso optimization problem is given as

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</munder>

<mfrac>

</mfrac>

</mrow>

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<msub>

</mrow>

</msub>

</mrow>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \mathrm {HSIC_{Lasso}} :\min _{\mathbf {x} }{\frac {1}{2}}\sum _{k,l=1}^{n}x_{k}x_{l}{\mbox{HSIC}}(f_{k},f_{l})-\sum _{k=1}^{n}x_{k}{\mbox{HSIC}}(f_{k},c)+\lambda \|\mathbf {x} \|_{1},\quad {\mbox{s.t.}}\ x_{1},\ldots ,x_{n}\geq 0,}</annotation>

</semantics>

</math><img src="./565c73fc34a225c00ee5fa197c4df8a686924656.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -3.338ex; width:91.274ex; height:7.176ex;" alt="{\displaystyle \mathrm {HSIC_{Lasso}} :\min _{\mathbf {x} }{\frac {1}{2}}\sum _{k,l=1}^{n}x_{k}x_{l}{\mbox{HSIC}}(f_{k},f_{l})-\sum _{k=1}^{n}x_{k}{\mbox{HSIC}}(f_{k},c)+\lambda \|\mathbf {x} \|_{1},\quad {\mbox{s.t.}}\ x_{1},\ldots ,x_{n}\geq 0,}" loading="lazy"></dd></dl>

where <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\mbox{HSIC}}(f_{k},c)={\mbox{tr}}({\bar {\mathbf {K} }}^{(k)}{\bar {\mathbf {L} }})}">

</mstyle>

</mrow>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<msup>

<mover>

</mrow>

</mover>

</mrow>

</mrow>

</msup>

<mover>

</mrow>

</mover>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\mbox{HSIC}}(f_{k},c)={\mbox{tr}}({\bar {\mathbf {K} }}^{(k)}{\bar {\mathbf {L} }})}</annotation>

</semantics>

</math><img src="./9e212b132611839868a75bdd767e80caef5d80ce.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:24.426ex; height:3.676ex;" alt="{\displaystyle {\mbox{HSIC}}(f_{k},c)={\mbox{tr}}({\bar {\mathbf {K} }}^{(k)}{\bar {\mathbf {L} }})}" loading="lazy"> is a kernel-based independence measure called the (empirical) Hilbert-Schmidt independence criterion (HSIC), <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\mbox{tr}}(\cdot )}">

</mstyle>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\mbox{tr}}(\cdot )}</annotation>

</semantics>

</math><img src="./aca2d8ae03a3a48a042072eee3f37f90347cb8cf.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:4.272ex; height:2.843ex;" alt="{\displaystyle {\mbox{tr}}(\cdot )}" loading="lazy"> denotes the <a href="Trace_(linear_algebra)" title="Trace (linear algebra)">trace</a>, <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \lambda }">

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \lambda }</annotation>

</semantics>

</math><img src="./b43d0ea3c9c025af1be9128e62a18fa74bedda2a.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:1.355ex; height:2.176ex;" alt="{\displaystyle \lambda }" loading="lazy"> is the regularization parameter, <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\bar {\mathbf {K} }}^{(k)}=\mathbf {\Gamma } \mathbf {K} ^{(k)}\mathbf {\Gamma } }">

<msup>

<mover>

</mrow>

</mover>

</mrow>

</mrow>

</msup>

</mrow>

<msup>

</mrow>

</mrow>

</msup>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\bar {\mathbf {K} }}^{(k)}=\mathbf {\Gamma } \mathbf {K} ^{(k)}\mathbf {\Gamma } }</annotation>

</semantics>

</math><img src="./3ecfa9dab9d79aa2eaa260133bab4b97f0b00947.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:15.239ex; height:3.176ex;" alt="{\displaystyle {\bar {\mathbf {K} }}^{(k)}=\mathbf {\Gamma } \mathbf {K} ^{(k)}\mathbf {\Gamma } }" loading="lazy"> and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\bar {\mathbf {L} }}=\mathbf {\Gamma } \mathbf {L} \mathbf {\Gamma } }">

<mover>

</mrow>

</mover>

</mrow>

</mrow>

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\bar {\mathbf {L} }}=\mathbf {\Gamma } \mathbf {L} \mathbf {\Gamma } }</annotation>

</semantics>

</math><img src="./3168805eb343f6e00a42e828a261399f52f6c6aa.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.338ex; width:9.532ex; height:2.509ex;" alt="{\displaystyle {\bar {\mathbf {L} }}=\mathbf {\Gamma } \mathbf {L} \mathbf {\Gamma } }" loading="lazy"> are input and output centered <a href="Gram_matrix" title="Gram matrix">Gram matrices</a>, <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K_{i,j}^{(k)}=K(u_{k,i},u_{k,j})}">

</mrow>

</mrow>

</msubsup>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K_{i,j}^{(k)}=K(u_{k,i},u_{k,j})}</annotation>

</semantics>

</math><img src="./440395184ec8f135fc425f66cb4604d6756d33cc.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.338ex; width:19.466ex; height:4.009ex;" alt="{\displaystyle K_{i,j}^{(k)}=K(u_{k,i},u_{k,j})}" loading="lazy"> and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle L_{i,j}=L(c_{i},c_{j})}">

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle L_{i,j}=L(c_{i},c_{j})}</annotation>

</semantics>

</math><img src="./2d7676e9fc7ed4e3f3d71ea802aac8229d627669.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:14.765ex; height:3.009ex;" alt="{\displaystyle L_{i,j}=L(c_{i},c_{j})}" loading="lazy"> are Gram matrices, <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle K(u,u')}">

<msup>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle K(u,u')}</annotation>

</semantics>

</math><img src="./a5af730a7c1eeaae84ac7a7fbef72dc242d2e705.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:8.253ex; height:3.009ex;" alt="{\displaystyle K(u,u')}" loading="lazy"> and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle L(c,c')}">

<msup>

</msup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle L(c,c')}</annotation>

</semantics>

</math><img src="./f2cf81e5c11793ede408adfbb5a317b625fe02af.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:7.124ex; height:3.009ex;" alt="{\displaystyle L(c,c')}" loading="lazy"> are kernel functions, <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mathbf {\Gamma } =\mathbf {I} _{m}-{\frac {1}{m}}\mathbf {1} _{m}\mathbf {1} _{m}^{T}}">

</mrow>

<msub>

</mrow>

</mrow>

</msub>

<mfrac>

</mfrac>

</mrow>

<msub>

</mrow>

</mrow>

</msub>

</mrow>

</mrow>

</mrow>

</msubsup>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \mathbf {\Gamma } =\mathbf {I} _{m}-{\frac {1}{m}}\mathbf {1} _{m}\mathbf {1} _{m}^{T}}</annotation>

</semantics>

</math><img src="./f2445cb6e01286ac4cf71bd9fe86dfedeb832a39.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.838ex; width:19.136ex; height:5.176ex;" alt="{\displaystyle \mathbf {\Gamma } =\mathbf {I} _{m}-{\frac {1}{m}}\mathbf {1} _{m}\mathbf {1} _{m}^{T}}" loading="lazy"> is the centering matrix, <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mathbf {I} _{m}}">

<msub>

</mrow>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \mathbf {I} _{m}}</annotation>

</semantics>

</math><img src="./dfb23acc2f2412a97f042bea6a93e7244d3dc380.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:2.689ex; height:2.509ex;" alt="{\displaystyle \mathbf {I} _{m}}" loading="lazy"> is the m-dimensional <a href="Identity_matrix" title="Identity matrix">identity matrix</a> (m: the number of samples), <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \mathbf {1} _{m}}">

<msub>

</mrow>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \mathbf {1} _{m}}</annotation>

</semantics>

</math><img src="./a44deb64836e8604e08fef06f4f84d5c470f2bb5.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:3.012ex; height:2.509ex;" alt="{\displaystyle \mathbf {1} _{m}}" loading="lazy"> is the m-dimensional vector with all ones, and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \|\cdot \|_{1}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \|\cdot \|_{1}}</annotation>

</semantics>

</math><img src="./3ec1a3a7b0d5372a6f2c6d12feda2cea000736bb.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:5.058ex; height:2.843ex;" alt="{\displaystyle \|\cdot \|_{1}}" loading="lazy"> is the <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \ell _{1}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \ell _{1}}</annotation>

</semantics>

</math><img src="./361ddd720474aa41cb05453e03424fb7999d3b02.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:2.024ex; height:2.509ex;" alt="{\displaystyle \ell _{1}}" loading="lazy">-norm. HSIC always takes a non-negative value, and is zero if and only if two random variables are statistically independent when a universal reproducing kernel such as the Gaussian kernel is used.

The HSIC Lasso can be written as

<msub>

</mrow>

</msub>

</mrow>

</mrow>

</munder>

<mfrac>

</mfrac>

</mrow>

<mrow>

<mrow>

<mover>

</mrow>

</mover>

</mrow>

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

<msup>

<mover>

</mrow>

</mover>

</mrow>

</mrow>

</msup>

</mrow>

</mrow>

</mrow>

</mrow>

</msubsup>

</mrow>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \mathrm {HSIC_{Lasso}} :\min _{\mathbf {x} }{\frac {1}{2}}\left\|{\bar {\mathbf {L} }}-\sum _{k=1}^{n}x_{k}{\bar {\mathbf {K} }}^{(k)}\right\|_{F}^{2}+\lambda \|\mathbf {x} \|_{1},\quad {\mbox{s.t.}}\ x_{1},\ldots ,x_{n}\geq 0,}</annotation>

</semantics>

</math><img src="./fac4ec7561efadfa3ffbca94d705438a0b2d4c67.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -3.171ex; width:67.866ex; height:7.843ex;" alt="{\displaystyle \mathrm {HSIC_{Lasso}} :\min _{\mathbf {x} }{\frac {1}{2}}\left\|{\bar {\mathbf {L} }}-\sum _{k=1}^{n}x_{k}{\bar {\mathbf {K} }}^{(k)}\right\|_{F}^{2}+\lambda \|\mathbf {x} \|_{1},\quad {\mbox{s.t.}}\ x_{1},\ldots ,x_{n}\geq 0,}" loading="lazy"></dd></dl>

where <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle \|\cdot \|_{F}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \|\cdot \|_{F}}</annotation>

</semantics>

</math><img src="./9e30c290fa57ceb0593f5dc01a5585ea1b8b54a4.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.838ex; width:5.467ex; height:2.843ex;" alt="{\displaystyle \|\cdot \|_{F}}" loading="lazy"> is the <a href="Frobenius_norm" class="mw-redirect" title="Frobenius norm">Frobenius norm</a>. The optimization problem is a Lasso problem, and thus it can be efficiently solved with a state-of-the-art Lasso solver such as the dual <a href="Augmented_Lagrangian_method" title="Augmented Lagrangian method">augmented Lagrangian method</a>.

<div class="mw-heading mw-heading2"><h2 id="Correlation_feature_selection">Correlation feature selection</h2></div>

The correlation feature selection (CFS) measure evaluates subsets of features on the basis of the following hypothesis: "Good feature subsets contain features highly correlated with the classification, yet uncorrelated to each other".<a href="#cite_note-42">[42]</a><a href="#cite_note-43">[43]</a> The following equation gives the merit of a feature subset S consisting of k features:

<msub>

</mrow>

<msub>

</mrow>

</msub>

</mrow>

</msub>

<mfrac>

<mrow>

<mover>

<msub>

</mrow>

</msub>

</mover>

</mrow>

<msqrt>

<mover>

<msub>

</mrow>

</msub>

</mover>

</mrow>

</msqrt>

</mfrac>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \mathrm {Merit} _{S_{k}}={\frac {k{\overline {r_{cf}}}}{\sqrt {k+k(k-1){\overline {r_{ff}}}}}}.}</annotation>

</semantics>

</math><img src="./16006751ea7a122dfbd6679f38ae3799088f3627.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -4.671ex; width:30.121ex; height:8.676ex;" alt="{\displaystyle \mathrm {Merit} _{S_{k}}={\frac {k{\overline {r_{cf}}}}{\sqrt {k+k(k-1){\overline {r_{ff}}}}}}.}" loading="lazy"></dd></dl>

Here, <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\overline {r_{cf}}}}">

<mover>

<msub>

</mrow>

</msub>

</mover>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\overline {r_{cf}}}}</annotation>

</semantics>

</math><img src="./c77995e20886e99a3c2ccd94a8adb0dadf08626b.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:3.012ex; height:3.009ex;" alt="{\displaystyle {\overline {r_{cf}}}}" loading="lazy"> is the average value of all feature-classification correlations, and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle {\overline {r_{ff}}}}">

<mover>

<msub>

</mrow>

</msub>

</mover>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle {\overline {r_{ff}}}}</annotation>

</semantics>

</math><img src="./19cc2ee1f97002ef2e4e9c4a61edd4af3fa72076.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:3.204ex; height:3.009ex;" alt="{\displaystyle {\overline {r_{ff}}}}" loading="lazy"> is the average value of all feature-feature correlations. The CFS criterion is defined as follows:

</mrow>

<msub>

</mrow>

</msub>

</mrow>

</munder>

<mrow>

<mfrac>

<mrow>

<msub>

<msub>

</mrow>

</msub>

</mrow>

</msub>

<msub>

<msub>

</mrow>

</msub>

</mrow>

</msub>

<msub>

<msub>

</mrow>

</msub>

</mrow>

</msub>

</mrow>

<msqrt>

<msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</msub>

<msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</msub>

<msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</msub>

</msqrt>

</mfrac>

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \mathrm {CFS} =\max _{S_{k}}\left[{\frac {r_{cf_{1}}+r_{cf_{2}}+\cdots +r_{cf_{k}}}{\sqrt {k+2(r_{f_{1}f_{2}}+\cdots +r_{f_{i}f_{j}}+\cdots +r_{f_{k}f_{k-1}})}}}\right].}</annotation>

</semantics>

</math><img src="./83568ac8d01463888fbfb13c56c9dd32a790699e.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -4.671ex; width:58.528ex; height:10.509ex;" alt="{\displaystyle \mathrm {CFS} =\max _{S_{k}}\left[{\frac {r_{cf_{1}}+r_{cf_{2}}+\cdots +r_{cf_{k}}}{\sqrt {k+2(r_{f_{1}f_{2}}+\cdots +r_{f_{i}f_{j}}+\cdots +r_{f_{k}f_{k-1}})}}}\right].}" loading="lazy"></dd></dl>

The <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle r_{cf_{i}}}">

<msub>

<msub>

</mrow>

</msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle r_{cf_{i}}}</annotation>

</semantics>

</math><img src="./d40aefc9429b8ed2adddefdbd4a5a90d30496141.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.005ex; width:3.423ex; height:2.343ex;" alt="{\displaystyle r_{cf_{i}}}" loading="lazy"> and <math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle r_{f_{i}f_{j}}}">

<msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle r_{f_{i}f_{j}}}</annotation>

</semantics>

</math><img src="./bccc2f330d679dac7bc9572fa33861f847eee2d2.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -1.338ex; width:4.231ex; height:2.676ex;" alt="{\displaystyle r_{f_{i}f_{j}}}" loading="lazy"> variables are referred to as correlations, but are not necessarily <a href="Pearson_product-moment_correlation_coefficient" class="mw-redirect" title="Pearson product-moment correlation coefficient">Pearson's correlation coefficient</a> or <a href="Spearman's_rank_correlation_coefficient" title="Spearman's rank correlation coefficient">Spearman's ρ</a>. Hall's dissertation uses neither of these, but uses three different measures of relatedness, <a href="Minimum_description_length" title="Minimum description length">minimum description length</a> (MDL), <a href="Mutual_Information" class="mw-redirect" title="Mutual Information">symmetrical uncertainty</a>, and <a href="Relief_(feature_selection)" title="Relief (feature selection)">relief</a>.

Let xi be the set membership <a href="Indicator_function" title="Indicator function">indicator function</a> for feature fi; then the above can be rewritten as an optimization problem:

</mrow>

<msup>

</mrow>

</msup>

</mrow>

</munder>

<mrow>

<mfrac>

<mrow>

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msup>

</mrow>

</msup>

</mrow>

<mrow>

</mrow>

</mrow>

</munderover>

<msub>

</mrow>

</msub>

</mrow>

</munder>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

<msub>

</mrow>

</msub>

</mrow>

</mfrac>

</mrow>

</mrow>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle \mathrm {CFS} =\max _{x\in \{0,1\}^{n}}\left[{\frac {(\sum _{i=1}^{n}a_{i}x_{i})^{2}}{\sum _{i=1}^{n}x_{i}+\sum _{i\neq j}2b_{ij}x_{i}x_{j}}}\right].}</annotation>

</semantics>

</math><img src="./9491bc46548bd4416952e59704e78388e8726480.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -3.171ex; width:43.615ex; height:7.509ex;" alt="{\displaystyle \mathrm {CFS} =\max _{x\in \{0,1\}^{n}}\left[{\frac {(\sum _{i=1}^{n}a_{i}x_{i})^{2}}{\sum _{i=1}^{n}x_{i}+\sum _{i\neq j}2b_{ij}x_{i}x_{j}}}\right].}" loading="lazy"></dd></dl>

The combinatorial problems above are, in fact, mixed 0–1 <a href="Linear_programming" title="Linear programming">linear programming</a> problems that can be solved by using <a href="Branch-and-bound_algorithm" class="mw-redirect" title="Branch-and-bound algorithm">branch-and-bound algorithms</a>.<a href="#cite_note-44">[44]</a>

<div class="mw-heading mw-heading2"><h2 id="Regularized_trees">Regularized trees</h2></div>

The features from a <a href="Decision_tree" title="Decision tree">decision tree</a> or a tree <a href="Ensemble_learning" title="Ensemble learning">ensemble</a> are shown to be redundant. A recent method called regularized tree<a href="#cite_note-DengRunger2012-45">[45]</a> can be used for feature subset selection. Regularized trees penalize using a variable similar to the variables selected at previous tree nodes for splitting the current node. Regularized trees only need build one tree model (or one tree ensemble model) and thus are computationally efficient.

Regularized trees naturally handle numerical and categorical features, interactions and nonlinearities. They are invariant to attribute scales (units) and insensitive to <a href="Outlier" title="Outlier">outliers</a>, and thus, require little <a href="Data_preprocessing" title="Data preprocessing">data preprocessing</a> such as <a href="Normalization_(statistics)" title="Normalization (statistics)">normalization</a>. Regularized random forest (RRF)<a href="#cite_note-RRF-46">[46]</a> is one type of regularized trees. The guided RRF is an enhanced RRF which is guided by the importance scores from an ordinary random forest.

<div class="mw-heading mw-heading2"><h2 id="Overview_on_metaheuristics_methods">Overview on metaheuristics methods</h2></div>

A <a href="Metaheuristic" title="Metaheuristic">metaheuristic</a> is a general description of an algorithm dedicated to solve difficult (typically <a href="NP-hardness" title="NP-hardness">NP-hard</a> problem) optimization problems for which there is no classical solving methods. Generally, a metaheuristic is a stochastic algorithm tending to reach a global optimum. There are many metaheuristics, from a simple local search to a complex global search algorithm.

<div class="mw-heading mw-heading3"><h3 id="Main_principles">Main principles</h3></div>

The feature selection methods are typically presented in three classes based on how they combine the selection algorithm and the model building.

<div class="mw-heading mw-heading4"><h4 id="Filter_method">Filter method</h4></div>

Filter type methods select variables regardless of the model. They are based only on general features like the correlation with the variable to predict. Filter methods suppress the least interesting variables. The other variables will be part of a classification or a regression model used to classify or to predict data. These methods are particularly effective in computation time and robust to overfitting.<a href="#cite_note-ReferenceA-47">[47]</a>

Filter methods tend to select redundant variables when they do not consider the relationships between variables. However, more elaborate features try to minimize this problem by removing variables highly correlated to each other, such as the Fast Correlation Based Filter (FCBF) algorithm.<a href="#cite_note-48">[48]</a>

<div class="mw-heading mw-heading4"><h4 id="Wrapper_method">Wrapper method</h4></div>

Wrapper methods evaluate subsets of variables which allows, unlike filter approaches, to detect the possible interactions amongst variables.<a href="#cite_note-M._Phuong,_Z_pages_301-309-49">[49]</a> The two main disadvantages of these methods are:

<ul><li>The increasing overfitting risk when the number of observations is insufficient.</li>

<li>The significant computation time when the number of variables is large.</li></ul>

<div class="mw-heading mw-heading4"><h4 id="Embedded_method">Embedded method</h4></div>

Embedded methods have been recently proposed that try to combine the advantages of both previous methods. A learning algorithm takes advantage of its own variable selection process and performs feature selection and classification simultaneously, such as the FRMT algorithm.<a href="#cite_note-50">[50]</a>

<div class="mw-heading mw-heading3"><h3 id="Application_of_feature_selection_metaheuristics">Application of feature selection metaheuristics</h3></div>

This is a survey of the application of feature selection metaheuristics lately used in the literature. This survey was realized by J. Hammon in her 2013 thesis.<a href="#cite_note-ReferenceA-47">[47]</a>

<th>Application</th>

<th>Algorithm</th>

<th>Approach</th>

<th>Classifier</th>

<th><a href="Evaluation_function" title="Evaluation function">Evaluation Function</a></th>

<th>Reference

</th></tr>

<tr>

<td>Feature Selection using Feature Similarity</td>

<td>Filter</td>

<td>Phuong 2005<a href="#cite_note-M._Phuong,_Z_pages_301-309-49">[49]</a>

</td></tr>

<tr>

<td><a href="Genetic_algorithm" title="Genetic algorithm">Genetic algorithm</a></td>

<td>Wrapper</td>

<td><a href="Decision_tree_learning" title="Decision tree learning">Decision Tree</a></td>

<td>Classification accuracy (10-fold)</td>

</td></tr>

<tr>

<td><a href="Hill_climbing" title="Hill climbing">Hill climbing</a></td>

<td>Filter + Wrapper</td>

<td><a href="Naive_Bayes_classifier" title="Naive Bayes classifier">Naive Bayesian</a></td>

<td>Predicted residual sum of squares</td>

</td></tr>

<tr>

<td><a href="Simulated_annealing" title="Simulated annealing">Simulated annealing</a></td>

<td>Naive bayesian</td>

<td>Classification accuracy (5-fold)</td>

<td>Ustunkar 2011<a href="#cite_note-53">[53]</a>

</td></tr>

<tr>

<td>Segments parole</td>

<td><a href="Ant_colony_optimization_algorithms" title="Ant colony optimization algorithms">Ant colony</a></td>

<td>Wrapper</td>

<td><a href="Artificial_Neural_Network" class="mw-redirect" title="Artificial Neural Network">Artificial Neural Network</a></td>

<td>Al-ani 2005

</td></tr>

<tr>

<td>Marketing</td>

<td>Simulated annealing</td>

<td>Wrapper</td>

<td>Regression</td>

<td>Meiri 2006<a href="#cite_note-54">[54]</a>

</td></tr>

<tr>

<td>Economics</td>

<td>Simulated annealing, genetic algorithm</td>

<td>Wrapper</td>

<td>Regression</td>

<td>Kapetanios 2007<a href="#cite_note-55">[55]</a>

</td></tr>

<tr>

<td>Spectral Mass</td>

<td>Genetic algorithm</td>

<td>Wrapper</td>

<td>Multiple Linear Regression, <a href="Partial_least_squares_regression" title="Partial least squares regression">Partial Least Squares</a></td>

<td><a href="Root-mean-square_error" class="mw-redirect" title="Root-mean-square error">root-mean-square error</a> of prediction</td>

<td>Broadhurst et al. 1997<a href="#cite_note-56">[56]</a>

</td></tr>

<tr>

<td><a href="Particle_swarm_optimization#Binary,_discrete,_and_combinatorial" title="Particle swarm optimization">Binary PSO</a> + <a href="Mutation_(genetic_algorithm)" class="mw-redirect" title="Mutation (genetic algorithm)">Mutation</a></td>

<td>Wrapper</td>

<td><a href="Decision_tree" title="Decision tree">Decision tree</a></td>

<td>weighted cost</td>

<td>Zhang 2014<a href="#cite_note-sciencedirect.com-24">[24]</a>

</td></tr>

<tr>

<td>Microarray</td>

<td><a href="Tabu_search" title="Tabu search">Tabu search</a> + <a href="Particle_swarm_optimization" title="Particle swarm optimization">PSO</a></td>

<td>Wrapper</td>

<td><a href="Support_Vector_Machine" class="mw-redirect" title="Support Vector Machine">Support Vector Machine</a>, <a href="K-nearest_neighbors_algorithm" title="K-nearest neighbors algorithm">K Nearest Neighbors</a></td>

<td><a href="Euclidean_Distance" class="mw-redirect" title="Euclidean Distance">Euclidean Distance</a></td>

<td>Chuang 2009<a href="#cite_note-57">[57]</a>

</td></tr>

<tr>

<td>Microarray</td>

<td>PSO + Genetic algorithm</td>

<td>Wrapper</td>

<td>Support Vector Machine</td>

<td>Classification accuracy (10-fold)</td>

</td></tr>

<tr>

<td>Microarray</td>

<td>Genetic algorithm + <a href="Iterated_local_search" title="Iterated local search">Iterated Local Search</a></td>

<td>Embedded</td>

<td>Support Vector Machine</td>

<td>Classification accuracy (10-fold)</td>

<td>Duval 2009<a href="#cite_note-B._Duval,_J_pages_201-208-59">[59]</a>

</td></tr>

<tr>

<td>Microarray</td>

<td>Iterated local search</td>

<td>Wrapper</td>

<td>Regression</td>

<td><a href="Posterior_probability" title="Posterior probability">Posterior Probability</a></td>

</td></tr>

<tr>

<td>Microarray</td>

<td>Genetic algorithm</td>

<td>Wrapper</td>

<td>K Nearest Neighbors</td>

<td>Classification accuracy (<a href="LOOCV" class="mw-redirect" title="LOOCV">Leave-one-out cross-validation</a>)</td>

<td>Jirapech-Umpai 2005<a href="#cite_note-61">[61]</a>

</td></tr>

<tr>

<td>Microarray</td>

<td><a href="Memetic_algorithm" title="Memetic algorithm">Hybrid genetic algorithm</a></td>

<td>Wrapper</td>

<td>K Nearest Neighbors</td>

<td>Classification accuracy (Leave-one-out cross-validation)</td>

</td></tr>

<tr>

<td>Microarray</td>

<td>Genetic algorithm</td>

<td>Wrapper</td>

<td>Support Vector Machine</td>

<td><a href="Sensitivity_and_specificity" title="Sensitivity and specificity">Sensitivity and specificity</a></td>

</td></tr>

<tr>

<td>Microarray</td>

<td>Genetic algorithm</td>

<td>Wrapper</td>

<td>All paired Support Vector Machine</td>

<td>Classification accuracy (Leave-one-out cross-validation)</td>

</td></tr>

<tr>

<td>Microarray</td>

<td>Genetic algorithm</td>

<td>Embedded</td>

<td>Support Vector Machine</td>

<td>Classification accuracy (10-fold)</td>

<td>Hernandez 2007<a href="#cite_note-65">[65]</a>

</td></tr>

<tr>

<td>Microarray</td>

<td>Genetic algorithm</td>

<td>Hybrid</td>

<td>Support Vector Machine</td>

<td>Classification accuracy (Leave-one-out cross-validation)</td>

<td>Huerta 2006<a href="#cite_note-66">[66]</a>

</td></tr>

<tr>

<td>Microarray</td>

<td>Genetic algorithm</td>

<td>Support Vector Machine</td>

<td>Classification accuracy (10-fold)</td>

</td></tr>

<tr>

<td>Microarray</td>

<td>Genetic algorithm</td>

<td>Wrapper</td>

<td>Support Vector Machine</td>

<td>EH-DIALL, CLUMP</td>

<td>Jourdan 2005<a href="#cite_note-68">[68]</a>

</td></tr>

<tr>

<td><a href="Alzheimer's_disease" title="Alzheimer's disease">Alzheimer's disease</a></td>

<td><a href="Welch's_t-test" title="Welch's t-test">Welch's t-test</a></td>

<td>Filter</td>

<td>Support vector machine</td>

<td>Classification accuracy (10-fold)</td>

<td>Zhang 2015<a href="#cite_note-69">[69]</a>

</td></tr>

<tr>

<td><a href="Computer_vision" title="Computer vision">Computer vision</a>

</td>

<td>Infinite Feature Selection

</td>

<td>Filter

</td>

<td>Independent

</td>

<td><a href="Average_Precision" class="mw-redirect" title="Average Precision">Average Precision</a>, <a href="ROC_curve" class="mw-redirect" title="ROC curve">ROC AUC</a>

</td>

<td>Roffo 2015<a href="#cite_note-70">[70]</a>

</td></tr>

<tr>

<td>Microarrays

</td>

<td>Eigenvector Centrality FS

</td>

<td>Filter

</td>

<td>Independent

</td>

<td>Average Precision, Accuracy, ROC AUC

</td>

<td>Roffo & Melzi 2016<a href="#cite_note-71">[71]</a>

</td></tr>

<tr>

<td>XML

</td>

<td>Symmetrical Tau (ST)

</td>

<td>Filter

</td>

<td>Structural Associative Classification

</td>

<td>Accuracy, Coverage

</td>

<td>Shaharanee & Hadzic 2014

</td></tr></tbody></table>

<div class="mw-heading mw-heading2"><h2 id="Feature_selection_embedded_in_learning_algorithms">Feature selection embedded in learning algorithms</h2></div>

Some learning algorithms perform feature selection as part of their overall operation. These include:

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle l_{1}}</annotation>

</semantics>

</math><img src="./29b25eeca673386d676f79dce674fe93040693eb.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:1.748ex; height:2.509ex;" alt="{\displaystyle l_{1}}" loading="lazy">⁠-regularization techniques, such as sparse regression, LASSO, and ⁠<math xmlns="http://www.w3.org/1998/Math/MathML" alttext="{\displaystyle l_{1}}">

<msub>

</mrow>

</msub>

</mstyle>

</mrow>

<annotation encoding="application/x-tex">{\displaystyle l_{1}}</annotation>

</semantics>

</math><img src="./29b25eeca673386d676f79dce674fe93040693eb.svg" class="mwe-math-fallback-image-inline mw-invert skin-invert" aria-hidden="true" style="vertical-align: -0.671ex; width:1.748ex; height:2.509ex;" alt="{\displaystyle l_{1}}" loading="lazy">⁠-SVM</li>

<li>Regularized trees,<a href="#cite_note-DengRunger2012-45">[45]</a> e.g. regularized random forest implemented in the RRF package<a href="#cite_note-RRF-46">[46]</a></li>

<li><a href="Decision_tree_learning" title="Decision tree learning">Decision tree</a><a href="#cite_note-72">[72]</a></li>

<li><a href="Memetic_algorithm" title="Memetic algorithm">Memetic algorithm</a></li>

<li><a href="Random_multinomial_logit" class="mw-redirect" title="Random multinomial logit">Random multinomial logit</a> (RMNL)</li>

<li><a href="Autoencoder" title="Autoencoder">Auto-encoding</a> networks with a bottleneck-layer</li>

<li><a href="Submodular_set_function" title="Submodular set function">Submodular</a> feature selection<a href="#cite_note-73">[73]</a><a href="#cite_note-74">[74]</a><a href="#cite_note-75">[75]</a></li>

<li>Local learning based feature selection.<a href="#cite_note-76">[76]</a> Compared with traditional methods, it does not involve any heuristic search, can easily handle multi-class problems, and works for both linear and nonlinear problems. It is also supported by a strong theoretical foundation. Numeric experiments showed that the method can achieve a close-to-optimal solution even when data contains >1M irrelevant features.</li>

<li>Recommender system based on feature selection.<a href="#cite_note-77">[77]</a> The feature selection methods are introduced into recommender system research.</li></ul>

<ul><li><a href="Cluster_analysis" title="Cluster analysis">Cluster analysis</a></li>

<li><a href="Data_mining" title="Data mining">Data mining</a></li>

<li><a href="Dimensionality_reduction" title="Dimensionality reduction">Dimensionality reduction</a></li>

<li><a href="Feature_extraction" class="mw-redirect" title="Feature extraction">Feature extraction</a></li>

<li><a href="Hyperparameter_optimization" title="Hyperparameter optimization">Hyperparameter optimization</a></li>

<li><a href="Model_selection" title="Model selection">Model selection</a></li>

<li><a href="Relief_(feature_selection)" title="Relief (feature selection)">Relief (feature selection)</a></li></ul>

<div class="mw-heading mw-heading2"><h2 id="References">References</h2></div>

/* start https://en.wikipedia.org/ */

.mw-parser-output .reflist{margin-bottom:0.5em;list-style-type:decimal}@media screen{.mw-parser-output .reflist{font-size:90%}}.mw-parser-output .reflist .references{font-size:100%;margin-bottom:0;list-style-type:inherit}.mw-parser-output .reflist-columns-2{column-width:30em}.mw-parser-output .reflist-columns-3{column-width:25em}.mw-parser-output .reflist-columns{margin-top:0.3em}.mw-parser-output .reflist-columns ol{margin-top:0}.mw-parser-output .reflist-columns li{page-break-inside:avoid;break-inside:avoid-column}.mw-parser-output .reflist-upper-alpha{list-style-type:upper-alpha}.mw-parser-output .reflist-upper-roman{list-style-type:upper-roman}.mw-parser-output .reflist-lower-alpha{list-style-type:lower-alpha}.mw-parser-output .reflist-lower-greek{list-style-type:lower-greek}.mw-parser-output .reflist-lower-roman{list-style-type:lower-roman}

/* end https://en.wikipedia.org/ */

</style><div class="reflist reflist-columns references-column-width" style="column-width: 30em;">

/* start https://en.wikipedia.org/ */

.mw-parser-output cite.citation{font-style:inherit;word-wrap:break-word}.mw-parser-output .citation q{quotes:"\"""\"""'""'"}.mw-parser-output .citation:target{background-color:rgba(0,127,255,0.133)}.mw-parser-output .id-lock-free.id-lock-free a{background:url("./mw/Lock-green.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-limited.id-lock-limited a,.mw-parser-output .id-lock-registration.id-lock-registration a{background:url("./mw/Lock-gray-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .id-lock-subscription.id-lock-subscription a{background:url("./mw/Lock-red-alt-2.svg")right 0.1em center/9px no-repeat}.mw-parser-output .cs1-ws-icon a{background:url("./mw/Wikisource-logo.svg")right 0.1em center/12px no-repeat}body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-free a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-limited a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-registration a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .id-lock-subscription a,body:not(.skin-timeless):not(.skin-minerva) .mw-parser-output .cs1-ws-icon a{background-size:contain;padding:0 1em 0 0}.mw-parser-output .cs1-code{color:inherit;background:inherit;border:none;padding:inherit}.mw-parser-output .cs1-hidden-error{display:none;color:var(--color-error,#d33)}.mw-parser-output .cs1-visible-error{color:var(--color-error,#d33)}.mw-parser-output .cs1-maint{display:none;color:#085;margin-left:0.3em}.mw-parser-output .cs1-kern-left{padding-left:0.2em}.mw-parser-output .cs1-kern-right{padding-right:0.2em}.mw-parser-output .citation .mw-selflink{font-weight:inherit}@media screen{.mw-parser-output .cs1-format{font-size:95%}html.skin-theme-clientpref-night .mw-parser-output .cs1-maint{color:#18911f}}@media screen and (prefers-color-scheme:dark){html.skin-theme-clientpref-os .mw-parser-output .cs1-maint{color:#18911f}}

/* end https://en.wikipedia.org/ */

</style><cite id="CITEREFGareth_JamesDaniela_WittenTrevor_HastieRobert_Tibshirani2013" class="citation book cs1">Gareth James; Daniela Witten; Trevor Hastie; Robert Tibshirani (2013). <a rel="nofollow" class="external text" href="http://www-bcf.usc.edu/~gareth/ISL/">An Introduction to Statistical Learning</a>. Springer. p. 204.</cite>

</li>

<li id="cite_note-2"><a href="#cite_ref-2">^</a> <cite id="CITEREFBrankMladenićGrobelnikLiu2011" class="citation cs2">Brank, Janez; Mladenić, Dunja; Grobelnik, Marko; Liu, Huan; Mladenić, Dunja; Flach, Peter A.; Garriga, Gemma C.; Toivonen, Hannu; Toivonen, Hannu (2011), <a rel="nofollow" class="external text" href="http://link.springer.com/10.1007/978-0-387-30164-8_306">"Feature Selection"</a>, in Sammut, Claude; Webb, Geoffrey I. (eds.), Encyclopedia of Machine Learning, Boston, MA: Springer US, pp. 402–406, <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1007%2F978-0-387-30164-8_306">10.1007/978-0-387-30164-8_306</a>, <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-0-387-30768-8</bdi>, retrieved 2021-07-13</cite>

</li>

<li id="cite_note-3"><a href="#cite_ref-3">^</a> <cite id="CITEREFKramer1991" class="citation journal cs1">Kramer, Mark A. (1991). <a rel="nofollow" class="external text" href="https://aiche.onlinelibrary.wiley.com/doi/abs/10.1002/aic.690370209">"Nonlinear principal component analysis using autoassociative neural networks"</a>. AIChE Journal. 37 (2): 233–243. <a href="Bibcode_(identifier)" class="mw-redirect" title="Bibcode (identifier)">Bibcode</a>:<a rel="nofollow" class="external text" href="https://ui.adsabs.harvard.edu/abs/1991AIChE..37..233K">1991AIChE..37..233K</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1002%2Faic.690370209">10.1002/aic.690370209</a>. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/1547-5905">1547-5905</a>.</cite>

</li>

<li id="cite_note-4"><a href="#cite_ref-4">^</a> <cite id="CITEREFKratsiosHyndman2021" class="citation journal cs1">Kratsios, Anastasis; Hyndman, Cody (2021). <a rel="nofollow" class="external text" href="http://jmlr.org/papers/v22/18-803.html">"NEU: A Meta-Algorithm for Universal UAP-Invariant Feature Representation"</a>. Journal of Machine Learning Research. 22 (92): 1–51. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/1533-7928">1533-7928</a>.</cite>

</li>

<li id="cite_note-5"><a href="#cite_ref-5">^</a> <cite id="CITEREFPerselloBruzzone2014" class="citation book cs1">Persello, Claudio; Bruzzone, Lorenzo (July 2014). <a rel="nofollow" class="external text" href="https://web.archive.org/web/20240414033437/https://ris.utwente.nl/ws/files/122945513/Persello2014relevant.pdf">"Relevant and invariant feature selection of hyperspectral images for domain generalization"</a> (PDF). <a rel="nofollow" class="external text" href="https://ris.utwente.nl/ws/files/122945513/Persello2014relevant.pdf">2014 IEEE Geoscience and Remote Sensing Symposium</a> (PDF). IEEE. pp. 3562–3565. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1109%2Figarss.2014.6947252">10.1109/igarss.2014.6947252</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-1-4799-5775-0</bdi>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:8368258">8368258</a>. Archived from <a rel="nofollow" class="external text" href="https://dx.doi.org/10.1109/igarss.2014.6947252">the original</a> on 2024-04-14. Retrieved 2023-07-22.</cite>

</li>

<li id="cite_note-6"><a href="#cite_ref-6">^</a> <cite id="CITEREFHinkleMuralidharanFletcherJoshi2012" class="citation book cs1">Hinkle, Jacob; Muralidharan, Prasanna; Fletcher, P. Thomas; Joshi, Sarang (2012). <a rel="nofollow" class="external text" href="https://link.springer.com/chapter/10.1007/978-3-642-33712-3_1">"Polynomial Regression on Riemannian Manifolds"</a>. In Fitzgibbon, Andrew; Lazebnik, Svetlana; Perona, Pietro; Sato, Yoichi; Schmid, Cordelia (eds.). Computer Vision – ECCV 2012. Lecture Notes in Computer Science. Vol. 7574. Berlin, Heidelberg: Springer. pp. 1–14. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/1201.2395">1201.2395</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1007%2F978-3-642-33712-3_1">10.1007/978-3-642-33712-3_1</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-3-642-33712-3</bdi>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:8849753">8849753</a>.</cite>

</li>

<li id="cite_note-7"><a href="#cite_ref-7">^</a> <cite id="CITEREFYarotsky2021" class="citation journal cs1">Yarotsky, Dmitry (2021-04-30). <a rel="nofollow" class="external text" href="https://doi.org/10.1007/s00365-021-09546-1">"Universal Approximations of Invariant Maps by Neural Networks"</a>. Constructive Approximation. 55: 407–474. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/1804.10306">1804.10306</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1007%2Fs00365-021-09546-1">10.1007/s00365-021-09546-1</a>. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/1432-0940">1432-0940</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:13745401">13745401</a>.</cite>

</li>

<li id="cite_note-8"><a href="#cite_ref-8">^</a> <cite id="CITEREFHaubergLauzePedersen2013" class="citation journal cs1">Hauberg, Søren; Lauze, François; Pedersen, Kim Steenstrup (2013-05-01). <a rel="nofollow" class="external text" href="https://doi.org/10.1007/s10851-012-0372-9">"Unscented Kalman Filtering on Riemannian Manifolds"</a>. Journal of Mathematical Imaging and Vision. 46 (1): 103–120. <a href="Bibcode_(identifier)" class="mw-redirect" title="Bibcode (identifier)">Bibcode</a>:<a rel="nofollow" class="external text" href="https://ui.adsabs.harvard.edu/abs/2013JMIV...46..103H">2013JMIV...46..103H</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1007%2Fs10851-012-0372-9">10.1007/s10851-012-0372-9</a>. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/1573-7683">1573-7683</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:8501814">8501814</a>.</cite>

</li>

<li id="cite_note-Bermingham-prolog-9"><a href="#cite_ref-Bermingham-prolog_9-0">^</a> <cite id="CITEREFKratsiosHyndman2021" class="citation journal cs1">Kratsios, Anastasis; Hyndman, Cody (June 8, 2021). <a rel="nofollow" class="external text" href="https://jmlr.org/papers/v22/18-803.html">"NEU: A Meta-Algorithm for Universal UAP-Invariant Feature Representation"</a>. <a href="Journal_of_Machine_Learning_Research" title="Journal of Machine Learning Research">Journal of Machine Learning Research</a>. 22: 10312. <a href="Bibcode_(identifier)" class="mw-redirect" title="Bibcode (identifier)">Bibcode</a>:<a rel="nofollow" class="external text" href="https://ui.adsabs.harvard.edu/abs/2015NatSR...510312B">2015NatSR...510312B</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1038%2Fsrep10312">10.1038/srep10312</a>. <a href="PMC_(identifier)" class="mw-redirect" title="PMC (identifier)">PMC</a> <a rel="nofollow" class="external text" href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4437376">4437376</a>. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/25988841">25988841</a>.</cite>

</li>

<li id="cite_note-guyon-intro-10">^ <a href="#cite_ref-guyon-intro_10-0">a</a> <a href="#cite_ref-guyon-intro_10-1">b</a> <a href="#cite_ref-guyon-intro_10-2">c</a> <cite id="CITEREFGuyonElisseeff2003" class="citation journal cs1">Guyon, Isabelle; Elisseeff, André (2003). <a rel="nofollow" class="external text" href="http://jmlr.csail.mit.edu/papers/v3/guyon03a.html">"An Introduction to Variable and Feature Selection"</a>. <a href="Journal_of_Machine_Learning_Research" title="Journal of Machine Learning Research">JMLR</a>. 3.</cite>

</li>

<li id="cite_note-textcat-11">^ <a href="#cite_ref-textcat_11-0">a</a> <a href="#cite_ref-textcat_11-1">b</a> <cite id="CITEREFYangPedersen1997" class="citation conference cs1">Yang, Yiming; Pedersen, Jan O. (1997). <a rel="nofollow" class="external text" href="http://www.surdeanu.info/mihai/teaching/ista555-spring15/readings/yang97comparative.pdf">A comparative study on feature selection in text categorization</a> (PDF). ICML.</cite>

</li>

<li id="cite_note-12"><a href="#cite_ref-12">^</a> <cite id="CITEREFUrbanowiczMeekerLaCavaOlson2018" class="citation journal cs1">Urbanowicz, Ryan J.; Meeker, Melissa; LaCava, William; Olson, Randal S.; Moore, Jason H. (2018). <a rel="nofollow" class="external text" href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6299836">"Relief-Based Feature Selection: Introduction and Review"</a>. Journal of Biomedical Informatics. 85: 189–203. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/1711.08421">1711.08421</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1016%2Fj.jbi.2018.07.014">10.1016/j.jbi.2018.07.014</a>. <a href="PMC_(identifier)" class="mw-redirect" title="PMC (identifier)">PMC</a> <a rel="nofollow" class="external text" href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC6299836">6299836</a>. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/30031057">30031057</a>.</cite>

</li>

<li id="cite_note-13"><a href="#cite_ref-13">^</a> <cite id="CITEREFForman2003" class="citation journal cs1">Forman, George (2003). <a rel="nofollow" class="external text" href="http://www.jmlr.org/papers/volume3/forman03a/forman03a.pdf">"An extensive empirical study of feature selection metrics for text classification"</a> (PDF). Journal of Machine Learning Research. 3: 1289–1305.</cite>

</li>

<li id="cite_note-14"><a href="#cite_ref-14">^</a> <cite id="CITEREFYishi_ZhangShujuan_LiTeng_WangZigang_Zhang2013" class="citation journal cs1">Yishi Zhang; Shujuan Li; Teng Wang; Zigang Zhang (2013). "Divergence-based feature selection for separate classes". Neurocomputing. 101 (4): 32–42. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1016%2Fj.neucom.2012.06.036">10.1016/j.neucom.2012.06.036</a>.</cite>

</li>

<li id="cite_note-15"><a href="#cite_ref-15">^</a> <cite id="CITEREFGuyon_I.Weston_J.Barnhill_S.Vapnik_V.2002" class="citation journal cs1">Guyon I.; Weston J.; Barnhill S.; Vapnik V. (2002). <a rel="nofollow" class="external text" href="https://doi.org/10.1023%2FA%3A1012487302797">"Gene selection for cancer classification using support vector machines"</a>. Machine Learning. 46 (1–3): 389–422. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1023%2FA%3A1012487302797">10.1023/A:1012487302797</a>.</cite>

</li>

<li id="cite_note-Bolasso-16"><a href="#cite_ref-Bolasso_16-0">^</a> <cite id="CITEREFBach2008" class="citation book cs1">Bach, Francis R (2008). "Bolasso". Proceedings of the 25th international conference on Machine learning - ICML '08. pp. 33–40. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1145%2F1390156.1390161">10.1145/1390156.1390161</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>9781605582054</bdi>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:609778">609778</a>.</cite>

</li>

<li id="cite_note-FeaLect-17"><a href="#cite_ref-FeaLect_17-0">^</a> <cite id="CITEREFZare2013" class="citation journal cs1">Zare, Habil (2013). <a rel="nofollow" class="external text" href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3549810">"Scoring relevancy of features based on combinatorial analysis of Lasso with application to lymphoma diagnosis"</a>. BMC Genomics. 14 (Suppl 1): S14. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1186%2F1471-2164-14-S1-S14">10.1186/1471-2164-14-S1-S14</a>. <a href="PMC_(identifier)" class="mw-redirect" title="PMC (identifier)">PMC</a> <a rel="nofollow" class="external text" href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3549810">3549810</a>. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/23369194">23369194</a>.</cite>

</li>

<li id="cite_note-18"><a href="#cite_ref-18">^</a> <cite id="CITEREFKai_HanYunhe_WangChao_ZhangChao_Li2018" class="citation conference cs1">Kai Han; Yunhe Wang; Chao Zhang; Chao Li; Chao Xu (2018). Autoencoder inspired unsupervised feature selection. IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP).</cite>

</li>

<li id="cite_note-19"><a href="#cite_ref-19">^</a> <cite id="CITEREFHazimehMazumderSaab2020" class="citation arxiv cs1">Hazimeh, Hussein; Mazumder, Rahul; Saab, Ali (2020). "Sparse Regression at Scale: Branch-and-Bound rooted in First-Order Optimization". <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/2004.06152">2004.06152</a> [<a rel="nofollow" class="external text" href="https://arxiv.org/archive/stat.CO">stat.CO</a>].</cite>

</li>

<li id="cite_note-20"><a href="#cite_ref-20">^</a> <cite id="CITEREFSoufanKleftogiannisKalnisBajic2015" class="citation journal cs1">Soufan, Othman; Kleftogiannis, Dimitrios; Kalnis, Panos; Bajic, Vladimir B. (2015-02-26). <a rel="nofollow" class="external text" href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4342225">"DWFS: A Wrapper Feature Selection Tool Based on a Parallel Genetic Algorithm"</a>. PLOS ONE. 10 (2): e0117988. <a href="Bibcode_(identifier)" class="mw-redirect" title="Bibcode (identifier)">Bibcode</a>:<a rel="nofollow" class="external text" href="https://ui.adsabs.harvard.edu/abs/2015PLoSO..1017988S">2015PLoSO..1017988S</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1371%2Fjournal.pone.0117988">10.1371/journal.pone.0117988</a>. <a href="ISSN_(identifier)" class="mw-redirect" title="ISSN (identifier)">ISSN</a> <a rel="nofollow" class="external text" href="https://search.worldcat.org/issn/1932-6203">1932-6203</a>. <a href="PMC_(identifier)" class="mw-redirect" title="PMC (identifier)">PMC</a> <a rel="nofollow" class="external text" href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4342225">4342225</a>. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/25719748">25719748</a>.</cite>

</li>

<li id="cite_note-21"><a href="#cite_ref-21">^</a> <cite id="CITEREFFigueroa2015" class="citation journal cs1">Figueroa, Alejandro (2015). <a rel="nofollow" class="external text" href="https://www.researchgate.net/publication/271911317">"Exploring effective features for recognizing the user intent behind web queries"</a>. Computers in Industry. 68: 162–169. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1016%2Fj.compind.2015.01.005">10.1016/j.compind.2015.01.005</a>.</cite>

</li>

<li id="cite_note-22"><a href="#cite_ref-22">^</a> <cite id="CITEREFFigueroaGuenter_Neumann2013" class="citation conference cs1">Figueroa, Alejandro; Guenter Neumann (2013). <a rel="nofollow" class="external text" href="https://www.researchgate.net/publication/259174469">Learning to Rank Effective Paraphrases from Query Logs for Community Question Answering</a>. AAAI.</cite>

</li>

<li id="cite_note-23"><a href="#cite_ref-23">^</a> <cite id="CITEREFFigueroaGuenter_Neumann2014" class="citation journal cs1">Figueroa, Alejandro; Guenter Neumann (2014). <a rel="nofollow" class="external text" href="https://www.researchgate.net/publication/260519271">"Category-specific models for ranking effective paraphrases in community Question Answering"</a>. Expert Systems with Applications. 41 (10): 4730–4742. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1016%2Fj.eswa.2014.02.004">10.1016/j.eswa.2014.02.004</a>. <a href="Hdl_(identifier)" class="mw-redirect" title="Hdl (identifier)">hdl</a>:<a rel="nofollow" class="external text" href="https://hdl.handle.net/10533%2F196878">10533/196878</a>.</cite>

</li>

<li id="cite_note-sciencedirect.com-24">^ <a href="#cite_ref-sciencedirect.com_24-0">a</a> <a href="#cite_ref-sciencedirect.com_24-1">b</a> <cite id="CITEREFZhangWangPhillips2014" class="citation journal cs1">Zhang, Y.; Wang, S.; Phillips, P. (2014). "Binary PSO with Mutation Operator for Feature Selection using Decision Tree applied to Spam Detection". Knowledge-Based Systems. 64: 22–31. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1016%2Fj.knosys.2014.03.015">10.1016/j.knosys.2014.03.015</a>.</cite>

</li>

<li id="cite_note-25"><a href="#cite_ref-25">^</a> F.C. Garcia-Lopez, M. Garcia-Torres, B. Melian, J.A. Moreno-Perez, J.M. Moreno-Vega. <a rel="nofollow" class="external text" href="https://pdfs.semanticscholar.org/ea5d/770e97b9330032e8713b0c105b523750a7c3.pdf">Solving feature subset selection problem by a Parallel Scatter Search</a>, European Journal of Operational Research, vol. 169, no. 2, pp. 477–489, 2006.

</li>

<li id="cite_note-26"><a href="#cite_ref-26">^</a> <cite id="CITEREFGarcía-TorresGómez-VelaDivinaPinto-Roa2021" class="citation book cs1">García-Torres, Miguel; Gómez-Vela, Francisco; Divina, Federico; Pinto-Roa, Diego P.; Noguera, José Luis Vázquez; Román, Julio C. Mello (2021). <a rel="nofollow" class="external text" href="https://dl.acm.org/doi/abs/10.1145/3449726.3459481">"Scatter search for high-dimensional feature selection using feature grouping"</a>. Proceedings of the Genetic and Evolutionary Computation Conference Companion. pp. 149–150. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1145%2F3449726.3459481">10.1145/3449726.3459481</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>9781450383516</bdi>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:235770316">235770316</a>.</cite>

</li>

<li id="cite_note-27"><a href="#cite_ref-27">^</a> M. Garcia-Torres. <a rel="nofollow" class="external text" href="https://link.springer.com/article/10.1007/s10732-025-09550-9">Feature selection for high-dimensional data using a multivariate search space reduction strategy based scatter search</a>, Journal of Heuristics, vol. 1, no 31, 2025.

</li>

<li id="cite_note-28"><a href="#cite_ref-28">^</a> F.C. Garcia-Lopez, M. Garcia-Torres, B. Melian, J.A. Moreno-Perez, J.M. Moreno-Vega. <a rel="nofollow" class="external text" href="https://web.archive.org/web/20190830132140/https://pdfs.semanticscholar.org/9428/2985d2c2ea4eb9f49846bedc12003a47db49.pdf">Solving Feature Subset Selection Problem by a Hybrid Metaheuristic</a>. In First International Workshop on Hybrid Metaheuristics, pp. 59–68, 2004.

</li>

<li id="cite_note-29"><a href="#cite_ref-29">^</a> M. Garcia-Torres, F. Gomez-Vela, B. Melian, J.M. Moreno-Vega. <a rel="nofollow" class="external text" href="https://www.researchgate.net/profile/Miguel_Garcia_Torres/publication/229763203_Parallel_Scatter_Search/links/5b2788a00f7e9be8bdaeb0d0/Parallel-Scatter-Search.pdf">High-dimensional feature selection via feature grouping: A Variable Neighborhood Search approach</a>, Information Sciences, vol. 326, pp. 102-118, 2016.

</li>

<li id="cite_note-30"><a href="#cite_ref-30">^</a> <cite id="CITEREFKraskovStögbauerAndrzejakGrassberger2003" class="citation journal cs1">Kraskov, Alexander; Stögbauer, Harald; Andrzejak, Ralph G; Grassberger, Peter (2003). "Hierarchical Clustering Based on Mutual Information". <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/q-bio/0311039">q-bio/0311039</a>. <a href="Bibcode_(identifier)" class="mw-redirect" title="Bibcode (identifier)">Bibcode</a>:<a rel="nofollow" class="external text" href="https://ui.adsabs.harvard.edu/abs/2003q.bio....11039K">2003q.bio....11039K</a>.</cite> <code class="cs1-code">{{cite journal}}</code>: Cite journal requires <code class="cs1-code">|journal=</code> (help)

</li>

<li id="cite_note-31"><a href="#cite_ref-31">^</a> <cite id="CITEREFAkaike1985" class="citation cs2"><a href="Hirotugu_Akaike" title="Hirotugu Akaike">Akaike, H.</a> (1985), "Prediction and entropy", in Atkinson, A. C.; <a href="Stephen_Fienberg" title="Stephen Fienberg">Fienberg, S. E.</a> (eds.), <a rel="nofollow" class="external text" href="https://apps.dtic.mil/dtic/tr/fulltext/u2/a120956.pdf">A Celebration of Statistics</a> (PDF), Springer, pp. 1–24, <a rel="nofollow" class="external text" href="https://web.archive.org/web/20190830132141/https://apps.dtic.mil/dtic/tr/fulltext/u2/a120956.pdf">archived</a> (PDF) from the original on August 30, 2019</cite>.

</li>

<li id="cite_note-32"><a href="#cite_ref-32">^</a> <cite id="CITEREFBurnhamAnderson2002" class="citation cs2">Burnham, K. P.; Anderson, D. R. (2002), <a rel="nofollow" class="external text" href="https://books.google.com/books?id=fT1Iu-h6E-oC">Model Selection and Multimodel Inference: A practical information-theoretic approach</a> (2nd ed.), <a href="Springer-Verlag" class="mw-redirect" title="Springer-Verlag">Springer-Verlag</a>, <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>9780387953649</bdi></cite>.

</li>

<li id="cite_note-33"><a href="#cite_ref-33">^</a> <cite id="CITEREFEinicke2018" class="citation journal cs1">Einicke, G. A. (2018). "Maximum-Entropy Rate Selection of Features for Classifying Changes in Knee and Ankle Dynamics During Running". IEEE Journal of Biomedical and Health Informatics. 28 (4): 1097–1103. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/2501.13750">2501.13750</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1109%2FJBHI.2017.2711487">10.1109/JBHI.2017.2711487</a>. <a href="Hdl_(identifier)" class="mw-redirect" title="Hdl (identifier)">hdl</a>:<a rel="nofollow" class="external text" href="https://hdl.handle.net/10810%2F68978">10810/68978</a>. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/29969403">29969403</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:49555941">49555941</a>.</cite>

</li>

<li id="cite_note-34"><a href="#cite_ref-34">^</a> <cite id="CITEREFAliferis2010" class="citation journal cs1">Aliferis, Constantin (2010). <a rel="nofollow" class="external text" href="http://jmlr.org/papers/volume11/aliferis10a/aliferis10a.pdf">"Local causal and markov blanket induction for causal discovery and feature selection for classification part I: Algorithms and empirical evaluation"</a> (PDF). Journal of Machine Learning Research. 11: 171–234.</cite>

</li>

<li id="cite_note-Brown-35">^ <a href="#cite_ref-Brown_35-0">a</a> <a href="#cite_ref-Brown_35-1">b</a> <a href="#cite_ref-Brown_35-2">c</a> <a href="#cite_ref-Brown_35-3">d</a> <cite id="CITEREFBrownPocockZhaoLuján2012" class="citation journal cs1">Brown, Gavin; Pocock, Adam; Zhao, Ming-Jie; Luján, Mikel (2012). <a rel="nofollow" class="external text" href="http://dl.acm.org/citation.cfm?id=2188385.2188387">"Conditional Likelihood Maximisation: A Unifying Framework for Information Theoretic Feature Selection"</a>. <a href="Journal_of_Machine_Learning_Research" title="Journal of Machine Learning Research">Journal of Machine Learning Research</a>. 13: 27–66.</cite><a rel="nofollow" class="external autonumber" href="http://www.jmlr.org/papers/volume13/brown12a/brown12a.pdf">[1]</a>

</li>

<li id="cite_note-36"><a href="#cite_ref-36">^</a> <cite id="CITEREFPengLongDing2005" class="citation journal cs1">Peng, H. C.; Long, F.; Ding, C. (2005). "Feature selection based on mutual information: criteria of max-dependency, max-relevance, and min-redundancy". <a href="IEEE_Transactions_on_Pattern_Analysis_and_Machine_Intelligence" title="IEEE Transactions on Pattern Analysis and Machine Intelligence">IEEE Transactions on Pattern Analysis and Machine Intelligence</a>. 27 (8): 1226–1238. <a href="CiteSeerX_(identifier)" class="mw-redirect" title="CiteSeerX (identifier)">CiteSeerX</a> <a rel="nofollow" class="external text" href="https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.63.5765">10.1.1.63.5765</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1109%2FTPAMI.2005.159">10.1109/TPAMI.2005.159</a>. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/16119262">16119262</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:206764015">206764015</a>.</cite> <a rel="nofollow" class="external text" href="http://home.penglab.com/proj/mRMR/index.htm">Program</a>

</li>

<li id="cite_note-docs.google-37"><a href="#cite_ref-docs.google_37-0">^</a> Nguyen, H., Franke, K., Petrovic, S. (2010). "Towards a Generic Feature-Selection Measure for Intrusion Detection", In Proc. International Conference on Pattern Recognition (ICPR), Istanbul, Turkey. <a rel="nofollow" class="external autonumber" href="https://www.researchgate.net/publication/220928649_Towards_a_Generic_Feature-Selection_Measure_for_Intrusion_Detection?ev=prf_pub">[2]</a>

</li>

<li id="cite_note-QPFS-38"><a href="#cite_ref-QPFS_38-0">^</a> <cite id="CITEREFRodriguez-LujanHuertaElkanSanta_Cruz2010" class="citation journal cs1">Rodriguez-Lujan, I.; Huerta, R.; Elkan, C.; Santa Cruz, C. (2010). <a rel="nofollow" class="external text" href="http://jmlr.csail.mit.edu/papers/volume11/rodriguez-lujan10a/rodriguez-lujan10a.pdf">"Quadratic programming feature selection"</a> (PDF). <a href="Journal_of_Machine_Learning_Research" title="Journal of Machine Learning Research">JMLR</a>. 11: 1491–1516.</cite>

</li>

<li id="cite_note-CMI-39">^ <a href="#cite_ref-CMI_39-0">a</a> <a href="#cite_ref-CMI_39-1">b</a> Nguyen X. Vinh, Jeffrey Chan, Simone Romano and James Bailey, "Effective Global Approaches for Mutual Information based Feature Selection". Proceedings of the 20th ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD'14), August 24–27, New York City, 2014. "<a rel="nofollow" class="external autonumber" href="http://people.eng.unimelb.edu.au/baileyj/papers/frp0038-Vinh.pdf">[3]</a>"

</li>

<li id="cite_note-40"><a href="#cite_ref-40">^</a> <cite id="CITEREFYangMoody2000" class="citation journal cs1">Yang, Howard Hua; Moody, John (2000). <a rel="nofollow" class="external text" href="https://papers.nips.cc/paper/1779-data-visualization-and-feature-selection-new-algorithms-for-nongaussian-data.pdf">"Data visualization and feature selection: New algorithms for nongaussian data"</a> (PDF). Advances in Neural Information Processing Systems: 687–693.</cite>

</li>

<li id="cite_note-HSICLasso-41"><a href="#cite_ref-HSICLasso_41-0">^</a> <cite id="CITEREFYamadaJitkrittumSigalXing2014" class="citation journal cs1">Yamada, M.; Jitkrittum, W.; Sigal, L.; Xing, E. P.; Sugiyama, M. (2014). "High-Dimensional Feature Selection by Feature-Wise Non-Linear Lasso". Neural Computation. 26 (1): 185–207. <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/1202.0515">1202.0515</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1162%2FNECO_a_00537">10.1162/NECO_a_00537</a>. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/24102126">24102126</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:2742785">2742785</a>.</cite>

</li>

<li id="cite_note-42"><a href="#cite_ref-42">^</a> <cite id="CITEREFHall1999" class="citation thesis cs1">Hall, M. (1999). <a rel="nofollow" class="external text" href="https://www.cs.waikato.ac.nz/~mhall/thesis.pdf">Correlation-based Feature Selection for Machine Learning</a> (PDF) (PhD thesis). University of Waikato.</cite>

</li>

<li id="cite_note-43"><a href="#cite_ref-43">^</a> <cite id="CITEREFSenliolGulgezenYuCataltepe2008" class="citation book cs1">Senliol, Baris; et al. (2008). "Fast Correlation Based Filter (FCBF) with a different search strategy". 2008 23rd International Symposium on Computer and Information Sciences. pp. 1–4. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1109%2FISCIS.2008.4717949">10.1109/ISCIS.2008.4717949</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-1-4244-2880-9</bdi>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:8398495">8398495</a>.</cite>

</li>

<li id="cite_note-44"><a href="#cite_ref-44">^</a> <cite id="CITEREFNguyenFrankePetrovic2009" class="citation journal cs1">Nguyen, Hai; Franke, Katrin; Petrovic, Slobodan (December 2009). <a rel="nofollow" class="external text" href="https://www.researchgate.net/publication/231175763">"Optimizing a class of feature selection measures"</a>. Proceedings of the NIPS 2009 Workshop on Discrete Optimization in Machine Learning: Submodularity, Sparsity & Polyhedra (DISCML). Vancouver, Canada.</cite>

</li>

<li id="cite_note-DengRunger2012-45">^ <a href="#cite_ref-DengRunger2012_45-0">a</a> <a href="#cite_ref-DengRunger2012_45-1">b</a> H. Deng, G. Runger, "<a rel="nofollow" class="external text" href="https://arxiv.org/abs/1201.1587">Feature Selection via Regularized Trees</a>", Proceedings of the 2012 International Joint Conference on Neural Networks (IJCNN), IEEE, 2012

</li>

<li id="cite_note-RRF-46">^ <a href="#cite_ref-RRF_46-0">a</a> <a href="#cite_ref-RRF_46-1">b</a> <a rel="nofollow" class="external text" href="https://cran.r-project.org/web/packages/RRF/index.html">RRF: Regularized Random Forest</a>, <a href="R_(programming_language)" title="R (programming language)">R</a> package on <a href="CRAN_(R_programming_language)" class="mw-redirect" title="CRAN (R programming language)">CRAN</a>

</li>

<li id="cite_note-ReferenceA-47">^ <a href="#cite_ref-ReferenceA_47-0">a</a> <a href="#cite_ref-ReferenceA_47-1">b</a> <cite id="CITEREFHamon2013" class="citation thesis cs1 cs1-prop-foreign-lang-source">Hamon, Julie (November 2013). <a rel="nofollow" class="external text" href="https://tel.archives-ouvertes.fr/tel-00920205">Optimisation combinatoire pour la sélection de variables en régression en grande dimension: Application en génétique animale</a> (Thesis) (in French). <a href="Lille_University_of_Science_and_Technology" title="Lille University of Science and Technology">Lille University of Science and Technology</a>.</cite>

</li>

<li id="cite_note-48"><a href="#cite_ref-48">^</a> <cite id="CITEREFYuLiu2003" class="citation journal cs1">Yu, Lei; Liu, Huan (August 2003). <a rel="nofollow" class="external text" href="https://www.aaai.org/Papers/ICML/2003/ICML03-111.pdf">"Feature selection for high-dimensional data: a fast correlation-based filter solution"</a> (PDF). ICML'03: Proceedings of the Twentieth International Conference on International Conference on Machine Learning: 856–863.</cite>

</li>

<li id="cite_note-M._Phuong,_Z_pages_301-309-49">^ <a href="#cite_ref-M._Phuong,_Z_pages_301-309_49-0">a</a> <a href="#cite_ref-M._Phuong,_Z_pages_301-309_49-1">b</a> T. M. Phuong, Z. Lin et R. B. Altman. <a rel="nofollow" class="external text" href="http://htsnp.stanford.edu/FSFS/TaggingSNP.pdf">Choosing SNPs using feature selection.</a> <a rel="nofollow" class="external text" href="https://web.archive.org/web/20160913211229/http://htsnp.stanford.edu/FSFS/TaggingSNP.pdf">Archived</a> 2016-09-13 at the <a href="Wayback_Machine" title="Wayback Machine">Wayback Machine</a> Proceedings / IEEE Computational Systems Bioinformatics Conference, CSB. IEEE Computational Systems Bioinformatics Conference, pages 301-309, 2005. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/16447987">16447987</a>.

</li>

<li id="cite_note-50"><a href="#cite_ref-50">^</a> <cite id="CITEREFSaghapourKermaniSehhati2017" class="citation journal cs1">Saghapour, E.; Kermani, S.; Sehhati, M. (2017). <a rel="nofollow" class="external text" href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5608217">"A novel feature ranking method for prediction of cancer stages using proteomics data"</a>. <a href="PLOS_ONE" class="mw-redirect" title="PLOS ONE">PLOS ONE</a>. 12 (9): e0184203. <a href="Bibcode_(identifier)" class="mw-redirect" title="Bibcode (identifier)">Bibcode</a>:<a rel="nofollow" class="external text" href="https://ui.adsabs.harvard.edu/abs/2017PLoSO..1284203S">2017PLoSO..1284203S</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1371%2Fjournal.pone.0184203">10.1371/journal.pone.0184203</a>. <a href="PMC_(identifier)" class="mw-redirect" title="PMC (identifier)">PMC</a> <a rel="nofollow" class="external text" href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5608217">5608217</a>. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/28934234">28934234</a>.</cite>

</li>

<li id="cite_note-51"><a href="#cite_ref-51">^</a> <cite id="CITEREFShahKusiak2004" class="citation journal cs1">Shah, S. C.; Kusiak, A. (2004). "Data mining and genetic algorithm based gene/SNP selection". Artificial Intelligence in Medicine. 31 (3): 183–196. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1016%2Fj.artmed.2004.04.002">10.1016/j.artmed.2004.04.002</a>. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/15302085">15302085</a>.</cite>

</li>

<li id="cite_note-52"><a href="#cite_ref-52">^</a> <cite id="CITEREFLongGianolaWeigel2011" class="citation journal cs1">Long, N.; Gianola, D.; Weigel, K. A (2011). "Dimension reduction and variable selection for genomic selection: application to predicting milk yield in Holsteins". Journal of Animal Breeding and Genetics. 128 (4): 247–257. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1111%2Fj.1439-0388.2011.00917.x">10.1111/j.1439-0388.2011.00917.x</a>. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/21749471">21749471</a>.</cite>

</li>

<li id="cite_note-53"><a href="#cite_ref-53">^</a> <cite id="CITEREFÜstünkarÖzöğür-AkyüzWeberFriedrich2012" class="citation journal cs1">Üstünkar, Gürkan; Özöğür-Akyüz, Süreyya; Weber, Gerhard W.; Friedrich, Christoph M.; Aydın Son, Yeşim (2012). "Selection of representative SNP sets for genome-wide association studies: A metaheuristic approach". Optimization Letters. 6 (6): 1207–1218. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1007%2Fs11590-011-0419-7">10.1007/s11590-011-0419-7</a>. <a href="Hdl_(identifier)" class="mw-redirect" title="Hdl (identifier)">hdl</a>:<a rel="nofollow" class="external text" href="https://hdl.handle.net/11511%2F30584">11511/30584</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:8075318">8075318</a>.</cite>

</li>

<li id="cite_note-54"><a href="#cite_ref-54">^</a> <cite id="CITEREFMeiriZahavi2006" class="citation journal cs1">Meiri, R.; Zahavi, J. (2006). "Using simulated annealing to optimize the feature selection problem in marketing applications". European Journal of Operational Research. 171 (3): 842–858. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1016%2Fj.ejor.2004.09.010">10.1016/j.ejor.2004.09.010</a>.</cite>

</li>

<li id="cite_note-55"><a href="#cite_ref-55">^</a> <cite id="CITEREFKapetanios2007" class="citation journal cs1">Kapetanios, G. (2007). "Variable Selection in Regression Models using Nonstandard Optimisation of Information Criteria". Computational Statistics & Data Analysis. 52 (1): 4–15. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1016%2Fj.csda.2007.04.006">10.1016/j.csda.2007.04.006</a>.</cite>

</li>

<li id="cite_note-56"><a href="#cite_ref-56">^</a> <cite id="CITEREFBroadhurstGoodacreJonesRowland1997" class="citation journal cs1">Broadhurst, D.; Goodacre, R.; Jones, A.; Rowland, J. J.; Kell, D. B. (1997). "Genetic algorithms as a method for variable selection in multiple linear regression and partial least squares regression, with applications to pyrolysis mass spectrometry". Analytica Chimica Acta. 348 (1–3): 71–86. <a href="Bibcode_(identifier)" class="mw-redirect" title="Bibcode (identifier)">Bibcode</a>:<a rel="nofollow" class="external text" href="https://ui.adsabs.harvard.edu/abs/1997AcAC..348...71B">1997AcAC..348...71B</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1016%2FS0003-2670%2897%2900065-2">10.1016/S0003-2670(97)00065-2</a>.</cite>

</li>

<li id="cite_note-57"><a href="#cite_ref-57">^</a> <cite id="CITEREFChuangYang2009" class="citation journal cs1">Chuang, L.-Y.; Yang, C.-H. (2009). "Tabu search and binary particle swarm optimization for feature selection using microarray data". Journal of Computational Biology. 16 (12): 1689–1703. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1089%2Fcmb.2007.0211">10.1089/cmb.2007.0211</a>. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/20047491">20047491</a>.</cite>

</li>

<li id="cite_note-58"><a href="#cite_ref-58">^</a> E. Alba, J. Garia-Nieto, L. Jourdan et E.-G. Talbi. <a rel="nofollow" class="external text" href="http://neo.lcc.uma.es/presentacionesCongreso/JMcec2007.pdf">Gene Selection in Cancer Classification using PSO-SVM and GA-SVM Hybrid Algorithms.</a> <a rel="nofollow" class="external text" href="https://web.archive.org/web/20160818135718/http://neo.lcc.uma.es/presentacionesCongreso/JMcec2007.pdf">Archived</a> 2016-08-18 at the <a href="Wayback_Machine" title="Wayback Machine">Wayback Machine</a> Congress on Evolutionary Computation, Singapore: Singapore (2007), 2007

</li>

<li id="cite_note-B._Duval,_J_pages_201-208-59"><a href="#cite_ref-B._Duval,_J_pages_201-208_59-0">^</a> B. Duval, J.-K. Hao et J. C. Hernandez Hernandez. <a rel="nofollow" class="external text" href="http://www.info.univ-angers.fr/pub/hao/papers/GECCO09.pdf">A memetic algorithm for gene selection and molecular classification of an cancer.</a> In Proceedings of the 11th Annual conference on Genetic and evolutionary computation, GECCO '09, pages 201-208, New York, NY, USA, 2009. ACM.

</li>

<li id="cite_note-60"><a href="#cite_ref-60">^</a> C. Hans, A. Dobra et M. West. <a rel="nofollow" class="external text" href="https://www.researchgate.net/profile/Adrian_Dobra/publication/228388856_Shotgun_Stochastic_Search_for_Large_p_Regression/links/02bfe5125185997d06000000.pdf">Shotgun stochastic search for 'large p' regression</a>. Journal of the American Statistical Association, 2007.

</li>

<li id="cite_note-61"><a href="#cite_ref-61">^</a> <cite id="CITEREFAitken2005" class="citation journal cs1">Aitken, S. (2005). <a rel="nofollow" class="external text" href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1181625">"Feature selection and classification for microarray data analysis: Evolutionary methods for identifying predictive genes"</a>. BMC Bioinformatics. 6 (1): 148. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1186%2F1471-2105-6-148">10.1186/1471-2105-6-148</a>. <a href="PMC_(identifier)" class="mw-redirect" title="PMC (identifier)">PMC</a> <a rel="nofollow" class="external text" href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1181625">1181625</a>. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/15958165">15958165</a>.</cite>

</li>

<li id="cite_note-62"><a href="#cite_ref-62">^</a> <cite id="CITEREFOhMoon2004" class="citation journal cs1">Oh, I. S.; Moon, B. R. (2004). "Hybrid genetic algorithms for feature selection". <a href="IEEE_Transactions_on_Pattern_Analysis_and_Machine_Intelligence" title="IEEE Transactions on Pattern Analysis and Machine Intelligence">IEEE Transactions on Pattern Analysis and Machine Intelligence</a>. 26 (11): 1424–1437. <a href="CiteSeerX_(identifier)" class="mw-redirect" title="CiteSeerX (identifier)">CiteSeerX</a> <a rel="nofollow" class="external text" href="https://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.467.4179">10.1.1.467.4179</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1109%2Ftpami.2004.105">10.1109/tpami.2004.105</a>. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/15521491">15521491</a>.</cite>

</li>

<li id="cite_note-63"><a href="#cite_ref-63">^</a> <cite id="CITEREFXuanGuoWangLiu2011" class="citation journal cs1">Xuan, P.; Guo, M. Z.; Wang, J.; Liu, X. Y.; Liu, Y. (2011). <a rel="nofollow" class="external text" href="https://doi.org/10.4238%2Fvol10-2gmr969">"Genetic algorithm-based efficient feature selection for classification of pre-miRNAs"</a>. Genetics and Molecular Research. 10 (2): 588–603. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.4238%2Fvol10-2gmr969">10.4238/vol10-2gmr969</a>. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/21491369">21491369</a>.</cite>

</li>

<li id="cite_note-64"><a href="#cite_ref-64">^</a> <cite id="CITEREFPeng2003" class="citation journal cs1">Peng, S. (2003). <a rel="nofollow" class="external text" href="https://doi.org/10.1016%2Fs0014-5793%2803%2901275-4">"Molecular classification of cancer types from microarray data using the combination of genetic algorithms and support vector machines"</a>. FEBS Letters. 555 (2): 358–362. <a href="Bibcode_(identifier)" class="mw-redirect" title="Bibcode (identifier)">Bibcode</a>:<a rel="nofollow" class="external text" href="https://ui.adsabs.harvard.edu/abs/2003FEBSL.555..358P">2003FEBSL.555..358P</a>. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1016%2Fs0014-5793%2803%2901275-4">10.1016/s0014-5793(03)01275-4</a>. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/14644442">14644442</a>.</cite>

</li>

<li id="cite_note-65"><a href="#cite_ref-65">^</a> <cite id="CITEREFHernandezDuvalHao2007" class="citation book cs1">Hernandez, J. C. H.; Duval, B.; Hao, J.-K. (2007). "A Genetic Embedded Approach for Gene Selection and Classification of Microarray Data". Evolutionary Computation, Machine Learning and Data Mining in Bioinformatics. EvoBIO 2007. Lecture Notes in Computer Science. Vol. 4447. Berlin: Springer Verlag. pp. 90–101. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1007%2F978-3-540-71783-6_9">10.1007/978-3-540-71783-6_9</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-3-540-71782-9</bdi>.</cite>

</li>

<li id="cite_note-66"><a href="#cite_ref-66">^</a> <cite id="CITEREFHuertaDuvalHao2006" class="citation book cs1">Huerta, E. B.; Duval, B.; Hao, J.-K. (2006). "A Hybrid GA/SVM Approach for Gene Selection and Classification of Microarray Data". Applications of Evolutionary Computing. EvoWorkshops 2006. Lecture Notes in Computer Science. Vol. 3907. pp. 34–44. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1007%2F11732242_4">10.1007/11732242_4</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-3-540-33237-4</bdi>.</cite>

</li>

<li id="cite_note-67"><a href="#cite_ref-67">^</a> <cite id="CITEREFMuniPalDas2006" class="citation journal cs1">Muni, D. P.; Pal, N. R.; Das, J. (2006). "Genetic programming for simultaneous feature selection and classifier design". IEEE Transactions on Systems, Man, and Cybernetics - Part B: Cybernetics. 36 (1): 106–117. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1109%2FTSMCB.2005.854499">10.1109/TSMCB.2005.854499</a>. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/16468570">16468570</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:2073035">2073035</a>.</cite>

</li>

<li id="cite_note-68"><a href="#cite_ref-68">^</a> <cite id="CITEREFJourdanDhaenensTalbi2005" class="citation journal cs1">Jourdan, L.; Dhaenens, C.; Talbi, E.-G. (2005). "Linkage disequilibrium study with a parallel adaptive GA". <a href="International_Journal_of_Foundations_of_Computer_Science" title="International Journal of Foundations of Computer Science">International Journal of Foundations of Computer Science</a>. 16 (2): 241–260. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1142%2FS0129054105002978">10.1142/S0129054105002978</a>.</cite>

</li>

<li id="cite_note-69"><a href="#cite_ref-69">^</a> <cite id="CITEREFZhangDongPhillipsWang2015" class="citation journal cs1">Zhang, Y.; Dong, Z.; Phillips, P.; Wang, S. (2015). <a rel="nofollow" class="external text" href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4451357">"Detection of subjects and brain regions related to Alzheimer's disease using 3D MRI scans based on eigenbrain and machine learning"</a>. Frontiers in Computational Neuroscience. 9: 66. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.3389%2Ffncom.2015.00066">10.3389/fncom.2015.00066</a>. <a href="PMC_(identifier)" class="mw-redirect" title="PMC (identifier)">PMC</a> <a rel="nofollow" class="external text" href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4451357">4451357</a>. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/26082713">26082713</a>.</cite>

</li>

<li id="cite_note-70"><a href="#cite_ref-70">^</a> <cite id="CITEREFRoffoMelziCristani2015" class="citation book cs1">Roffo, G.; Melzi, S.; Cristani, M. (2015-12-01). "Infinite Feature Selection". 2015 IEEE International Conference on Computer Vision (ICCV). pp. 4202–4210. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1109%2FICCV.2015.478">10.1109/ICCV.2015.478</a>. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>978-1-4673-8391-2</bdi>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:3223980">3223980</a>.</cite>

</li>

<li id="cite_note-71"><a href="#cite_ref-71">^</a> <cite id="CITEREFRoffoMelzi2016" class="citation web cs1">Roffo, Giorgio; Melzi, Simone (September 2016). <a rel="nofollow" class="external text" href="http://www.di.uniba.it/~loglisci/NFmcp2016/NFmcp2016_paper_13.pdf">"Features Selection via Eigenvector Centrality"</a> (PDF). NFmcp2016. Retrieved 12 November 2016.</cite>

</li>

<li id="cite_note-72"><a href="#cite_ref-72">^</a> R. Kohavi and G. John, "<a rel="nofollow" class="external text" href="https://ai.stanford.edu/~ronnyk/wrappersPrint.pdf">Wrappers for feature subset selection</a>", <a href="Artificial_Intelligence_(journal)" title="Artificial Intelligence (journal)">Artificial intelligence</a> 97.1-2 (1997): 273-324

</li>

<li id="cite_note-73"><a href="#cite_ref-73">^</a> <cite id="CITEREFDasKempe2011" class="citation arxiv cs1">Das, Abhimanyu; Kempe, David (2011). "Submodular meets Spectral: Greedy Algorithms for Subset Selection, Sparse Approximation and Dictionary Selection". <a href="ArXiv_(identifier)" class="mw-redirect" title="ArXiv (identifier)">arXiv</a>:<a rel="nofollow" class="external text" href="https://arxiv.org/abs/1102.3975">1102.3975</a> [<a rel="nofollow" class="external text" href="https://arxiv.org/archive/stat.ML">stat.ML</a>].</cite>

</li>

<li id="cite_note-74"><a href="#cite_ref-74">^</a> Liu et al., <a rel="nofollow" class="external text" href="http://melodi.ee.washington.edu/~bilmes/mypubs/liu-submodfeature2013-icassp.pdf">Submodular feature selection for high-dimensional acoustic score spaces</a> <a rel="nofollow" class="external text" href="https://web.archive.org/web/20151017122628/http://melodi.ee.washington.edu/~bilmes/mypubs/liu-submodfeature2013-icassp.pdf">Archived</a> 2015-10-17 at the <a href="Wayback_Machine" title="Wayback Machine">Wayback Machine</a>

</li>

<li id="cite_note-75"><a href="#cite_ref-75">^</a> Zheng et al., <a rel="nofollow" class="external text" href="http://papers.nips.cc/paper/5565-deep-convolutional-neural-network-for-image-deconvolution">Submodular Attribute Selection for Action Recognition in Video</a> <a rel="nofollow" class="external text" href="https://web.archive.org/web/20151118001059/http://papers.nips.cc/paper/5565-deep-convolutional-neural-network-for-image-deconvolution">Archived</a> 2015-11-18 at the <a href="Wayback_Machine" title="Wayback Machine">Wayback Machine</a>

</li>

<li id="cite_note-76"><a href="#cite_ref-76">^</a> <cite id="CITEREFSunTodorovicGoodison2010" class="citation journal cs1">Sun, Y.; Todorovic, S.; Goodison, S. (2010). <a rel="nofollow" class="external text" href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3445441">"Local-Learning-Based Feature Selection for High-Dimensional Data Analysis"</a>. <a href="IEEE_Transactions_on_Pattern_Analysis_and_Machine_Intelligence" title="IEEE Transactions on Pattern Analysis and Machine Intelligence">IEEE Transactions on Pattern Analysis and Machine Intelligence</a>. 32 (9): 1610–1626. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1109%2Ftpami.2009.190">10.1109/tpami.2009.190</a>. <a href="PMC_(identifier)" class="mw-redirect" title="PMC (identifier)">PMC</a> <a rel="nofollow" class="external text" href="https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3445441">3445441</a>. <a href="PMID_(identifier)" class="mw-redirect" title="PMID (identifier)">PMID</a> <a rel="nofollow" class="external text" href="https://pubmed.ncbi.nlm.nih.gov/20634556">20634556</a>.</cite>

</li>

<li id="cite_note-77"><a href="#cite_ref-77">^</a> D.H. Wang, Y.C. Liang, D.Xu, X.Y. Feng, R.C. Guan(2018), "<a rel="nofollow" class="external text" href="https://www.sciencedirect.com/science/article/pii/S0950705118302107">A content-based recommender system for computer science publications</a>", <a href="Knowledge-Based_Systems" class="mw-redirect" title="Knowledge-Based Systems">Knowledge-Based Systems</a>, 157: 1-9

</li>

</ol></div>

<div class="mw-heading mw-heading2"><h2 id="Further_reading">Further reading</h2></div>

<ul><li><cite id="CITEREFGuyonElisseeff2003" class="citation journal cs1">Guyon, Isabelle; Elisseeff, Andre (2003). <a rel="nofollow" class="external text" href="http://www.jmlr.org/papers/v3/guyon03a.html">"An Introduction to Variable and Feature Selection"</a>. Journal of Machine Learning Research. 3: 1157–1182.</cite></li>

<li><cite id="CITEREFHarrell2001" class="citation book cs1">Harrell, F. (2001). Regression Modeling Strategies. Springer. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>0-387-95232-2</bdi>.</cite></li>

<li><cite id="CITEREFLiuMotoda1998" class="citation book cs1">Liu, Huan; Motoda, Hiroshi (1998). <a rel="nofollow" class="external text" href="https://books.google.com/books?id=aaDbBwAAQBAJ">Feature Selection for Knowledge Discovery and Data Mining</a>. Springer. <a href="ISBN_(identifier)" class="mw-redirect" title="ISBN (identifier)">ISBN</a> <bdi>0-7923-8198-X</bdi>.</cite></li>

<li><cite id="CITEREFLiuYu2005" class="citation journal cs1">Liu, Huan; Yu, Lei (2005). "Toward Integrating Feature Selection Algorithms for Classification and Clustering". IEEE Transactions on Knowledge and Data Engineering. 17 (4): 491–502. <a href="Doi_(identifier)" class="mw-redirect" title="Doi (identifier)">doi</a>:<a rel="nofollow" class="external text" href="https://doi.org/10.1109%2FTKDE.2005.66">10.1109/TKDE.2005.66</a>. <a href="S2CID_(identifier)" class="mw-redirect" title="S2CID (identifier)">S2CID</a> <a rel="nofollow" class="external text" href="https://api.semanticscholar.org/CorpusID:1607600">1607600</a>.</cite></li></ul>

<div class="mw-heading mw-heading2"><h2 id="External_links">External links</h2></div>

<ul><li><a rel="nofollow" class="external text" href="https://web.archive.org/web/20120511162342/http://featureselection.asu.edu/software.php">Feature Selection Package, Arizona State University (Matlab Code)</a></li>

<li><a rel="nofollow" class="external text" href="http://www.clopinet.com/isabelle/Projects/NIPS2003/">NIPS challenge 2003</a> (see also <a href="Conference_on_Neural_Information_Processing_Systems" title="Conference on Neural Information Processing Systems">NIPS</a>)</li>

<li><a rel="nofollow" class="external text" href="http://paul.luminos.nl/documents/show_document.php?d=198">Naive Bayes implementation with feature selection in Visual Basic</a> <a rel="nofollow" class="external text" href="https://web.archive.org/web/20090214023159/http://paul.luminos.nl/documents/show_document.php?d=198">Archived</a> 2009-02-14 at the <a href="Wayback_Machine" title="Wayback Machine">Wayback Machine</a> (includes executable and source code)</li>

<li><a rel="nofollow" class="external text" href="http://home.penglab.com/proj/mRMR/index.htm">Minimum-redundancy-maximum-relevance (mRMR) feature selection program</a></li>

<li><a rel="nofollow" class="external text" href="http://mloss.org/software/view/386/">FEAST</a> (Open source Feature Selection algorithms in C and MATLAB)</li></ul></div><div><div class="zim-footer">

This article is issued from <a class="external text" title="Last edited on 2025-08-05" href="https://en.wikipedia.org/wiki/?title=Feature_selection&oldid=1304314871">Wikipedia</a>. The text is available under <a class="external text" href="https://creativecommons.org/licenses/by-sa/4.0/deed.en">Creative Commons Attribution-Share Alike 4.0</a> unless otherwise noted. Additional terms may apply for the media files.

</div>

</div></div>

</div>

</main>

</div>

</body></html>